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INTRODUCTION 



THE CONTEXT 

This guidebook is designed to help individuals in the business community plan, develop, 
implement, and evaluate high quality workplace literacy programs for their companies. The 
need for such programs is inextricably linked to the increasingly global marketplace in which 
American firms are forced to compete. Without a solid foundation in literacy skills, 
employees have difficulty adapting to the new technologies or functioning effectively in the 
high performance work organizations that companies are adopting in order to compete and 
survive. 

Businesses of all types, particularly those in manufacturing, must recognize and deal with the 
pressures and challenges they face in today's global economy. Rapid changes in technology 
and increased international competition have taken their toll. American manufacturers can 
either lose their competitive edge, or rise up and do battle. The survivors will be those 
businesses that come up with new approaches and strategies that are both innovative and cost 
effective. 

Often new strategies and approaches are blocked by complex problems that include the 
difficulty of altering the way we are accustomed to doing our work, the changing 
demographics and educational backgrounds of workers in America, and the failure of our 
education system to keep up with the continuing education needs of adults. The challenge 
and the responsibility for addressing these barriers to improvement fall to the workplace, 
where success and profit depend on the skills of the men and women who actually do the 
work. 

Never before have American manufacturing firms been faced with such tremendous market 
pressures. In order to remain competitive, successful manufacturers have adopted 
approaches that are characterized by Massachusetts Institute of Technology researchers as 
"lean manufacturing." This involves; 

• The development of skilled worker teams; 

• The use of flexible, automated machinery; 

• The production of customized products to exacting standards quickly and at amazingly 
low cost; and 

• The use of "precision strike" training to maximize the abilities of the most important 
ingredient of all the people. 
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Precision strike training is training designed and delivered around specifically identified 
training needs. It can be contrasted with the more traditional "saturation" training, which 
provides all employees with similar training, differentiating neither on the basis of the skill 
requirements of individuals' jobs nor the skills individuals already possess. 

Lean manufacturing and precision strike training go hand in hand. In a lean manufacturing 
environment, the inefficiency of saturation training would carry with it unacceptably high 
costs. Lean manufacturing environments require precision strike training — training designed 
to fill the gaps between existing worker skill levels and the skill levels required to perform 
current and future jobs. 

Most companies have a workplace literacy problem, though many do not recognize it. A 
large number of employees without high school diplomas is a good indicator of a problem, 
though high school diplomas do not guarantee adequate skills. Even if companies do not 
seem to have workplace literacy problems, such problems may surface as the companies try 
to implement new procedures that require higher-level math skills, such as statistical process 
control, or initiate cross training, teamwork, or quality initiatives, which require higher 
verbal skills. 

CERT AENTEED' S EXPERIENCES 

For CertainTeed, a building materials manufacturer with asphalt roofing plants in Minnesota, 
North Carolina, and Ohio, workplace literacy was a natural outgrowth of management's 
commitment to "lifelong learning," the CEO's desire to be an "education president," and the 
vice president of manufacturing's interest in promoting from within. CertainTeed was most 
interested in investing in a pilot literacy program that would be carefully evaluated to 
determine whether its benefits outweighed its costs, and then only expanded if it proved 
warranted. 

CertainTeed eventually launched two workplace literacy pilot programs: 

• CertainTeed's Skills and Knowledge in Lifelong Learning (S.K.I.L.L.) workplace 
literacy program at the Oxford, North Carolina, roofing shingles plant; and 

• A special project, "Precision-Strike Training in a Lean Manufacturing Environment/' 
funded by the National Institute for Literacy and implemented through a partnership 
consisting of CertainTeed, the National Alliance of Business, and Vance-Granville 
Community College. 

Both projects emphasized contextual learning and problem solving, two big pluses with 
management at CertainTeed. 



the S.K.I.L.L. Program 



The S.K.I.L.L. pilot program was officially started in September, 1991, at CertainTeed's 
Oxford, North Carolina, plant, after a research period during which the training needs of the 
roofing products group were identified. S.K.I.L.L. was designed as an 18 month pilot 
program, to be expanded if an evaluation showed an expansion was merited. 

The foundation of the S.K.I.L.L. program was the establishment of an on-site learning 
center, equipped with the capacity to deliver training in the context of the CertainTeed 
workplace, using computer $nd interactive video-based learning, classroom training, and on- 
the-job training (OJT). The learning center was open 24 hours daily to accommodate 
workers on different shifts, and was staffed about 40 hours weekly. 

S.K.I.L.L. was designed as an open entry-open exit, voluntary program. As part of 
S.K.I.L.L., jobs were classified at one of five levels. Current employees could choose to 
enroll in training at the level in which their current jobs fell or at one level above that. 
However, they had to successfully complete the training level in which their current job fell 
before enrolling in the training at the next higher level. All new employees were required to 
enroll in the S.K.I.L.L. program following a week of orientation. 

As part of the S.K.I.L.L. program, CertainTeed hired a training manager and contributed to 
the salaries of Vance-Granville Community College instructors who staffed the learning 
center. CertainTeed also paid for one half of the time participants spent working on the 
S.K.I.L.L. program. 

Early in S.K.I.L.L. 's implementation, a consultant was hired to conduct a literacy audit. 
One recommendation from the audit was to include more contextual material. As a result, 
Precision Strike was born. 

Precision Strike Training 

Precision Strike was, in essence, a "pilot within a pilot." It was designed to compare the 
learning gains and job performance gains from providing one crew with an enhanced 
S.K.I.L.L. program with the experiences of two other crews whose members may or may 
not have been participating in the regular S.K.I.L.L. program. 

The enhanced curriculum consisted of a functional context workbook customized through 
literacy task analyses of the targeted jobs, with a unique workbook for each type of job. 
This workbook supplemented regular S.K.I.L.L. material. The individuals from the crew 
participating in the Precision Strike training program had to spend at least three hours per 
week over a period of 16 weeks working on their training materials. 
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THE GUIDEBOOK 



This guidebook draws on the two CertainTeed workplace literacy training programs - 
S.K.I.L.L, and Precision Strike, It is unique because it combines best practices from the 
literature with real-life, hands-on experiences at CertainTeed Corporation's Oxford, North 
Carolina, roofing manufacturing plant, 

CertainTeed's Precision Strike training program was an enhancement of S.K.I.L.L - it 
incorporated all of the features of the S.KT.L.L, program as well as those developed solely 
for the Precision Strike pilot. Therefore, duplicating CertainTeed's Precision Strike training 
program requires undertaking the planning, development, and operational tasks associated 
with both S.K.I.L.L. and Precision Strike. 

Since both S.K.I.L.L. and Precision Strike tasks must be accomplished in order to create a 
new precision strike training program, this workplace literacy guide does not differentiate 
between CertainTeed's S.K.I.L.L. and Precision Strike programs, either in laying out the 
tasks and steps required to duplicate this project, or in describing the activities carried out at 
CertainTeed. Instead, everything is attributed to Precision Strike. 

This guidebook is organized around training tasks. These tasks provide an approach and 
framework for developing training in manufacturing settings. Each task is followed by a 
series of steps to be accomplished. Together these tasks and steps take you through a 
systematic, yet "doable," process for establishing a functional context workplace literacy 
program. Illustrative material is provided directly from CertainTeed's experiences. You 
may well find this material to be the most useful, because it provides a wealth of practical, 
"how-to-do-it" information. 

While the guidebook is systematic, the tasks and steps it presents do not have to be carried 
out in the exact order presented. Companies using this guidebook should feel free to 
perform some tasks and steps simultaneously instead of sequentially, or even shuffle the 
order of some, in order to meet their own needs and circumstances. The key to a successful 
precision strike training program is to carefully target training to meet clearly defined skill 
needs. Within tnat simple maxim, a great deal of flexibility can be practiced. 
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L GROUNDWORK 



TASK 1: SECURE COMMITMENT, SUPPORT, AND RESOURCES 

The success of a precision strike training program may be directly dependent upon the level 
of commitment, support, and resources that can be generated and maintained. To get this 
program off the ground, either you must serve as a change agent or advocate, or you must 
recruit someone else to take on this role. In either case, the advocate must then: 

• Pull together information that clearly demonstrates that a workplace literacy problem 
exists; 

• Make the case to the executive leadership that they need to address this problem; 

• Build widespread ownership throughout the company for a workplace literacy 
program; and 

• Recruit program partners to help develop and deliver the program. 
Step 1: Get a change agent or advocate 

In the earliest stages of your program's development, you need someone or some group to 
serve in the role of change agent or advocate. The advocate, which can be you, should have 
the clout and expertise to marshall interest in the training project and ensure that all the 
necessary activities (planning, development, implementation, evaluation) are carried out. 

The advocate should also take the lead in identifying other experts who can help in the 
development and implementation of the training program. This may involve developing in- 
house competence at the plant level, buying expertise from consultants, or finding other 
valuable resources, such as community college instructors or state literacy experts. 

Step 2: Gather information on the scope of the problem 

The first critical step in putting together a precision strike training intervention program is 
recognizing and understanding that workforce skills deficits are a business problem. No 
company can afford to ignore skills problems, because a key factor in achieving company 
goals is the quality of current and future workers. 

Workplace literacy training cannot occur unless there is a recognition and understanding by 
company leaders that a problem exists. Getting a handle on the scale of this problem 
involves: 
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• Determining whether there is a mismatch between the skills of your company's 
workforce and your current skills needs; 

• Assessing the effect of your company's goals and strategies on your labor needs (both 
size and skills); and 

• Determining how changes in the international marketplace — both product demand and 
labor supply -- will affect your company's labor needs and supply. 

To determine whether there currently is a mismatch between your employees' skills and the 
skills you need, you should look at a few common indicators of workplace literacy problems: 

• Fewer qualified applicants and hires; 

• Performance mistakes that cost time, money, or customers, or result in accidents; 

• Lack of success in technical training or upgrading; and 

• Slow progress in the introduction and implementation of work teams. 

Assessing the impact your company's goals and strategies will have on your labor needs is 
more complex. You must first examine your company's current and future business plans, 
goals, and objectives, and then determine their implications for product quality and price; 
production efficiency and economy; and the company's technology, equipment, and 
instrumentation needs. With this information, you can begin to assess the skills your 
employees must have in the future and whether you will need to begin upgrading their skills 
now. 

Finally, it is important to look at the bigger picture and determine how the larger economy 
will affect both your labor demand and supply. This involves gauging economic conditions, 
sizing up the demand for higher quality products and/or a more diversified product line, 
identifying pressures from global competition, appraising the size and quality of the future 
labor force, and then applying the knowledge learned about the marketplace to your company 
and assessing how your local workplace will be affected. 

Step 3: Get commitment from the executive leadership 

Once you have a good understanding of the extent of the problem, market your workplace 
literacy training plan to the business leadership. Because developing and implementing a 
precision strike training program will require a heavy investment in time and money, the 
company's leaders must be convinced that, in the end, the investment will pay off. 
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First, do your homework. Chances are you will be competing with other training interests 
and priorities. It is extremely important to be well informed and well versed in your training 
plan — its features, benefits, and costs. 

Second, educate and inform leadership in a manner that helps them recognize that a problem 
exists. Promote workplace literacy training for what it is - an intervention or program to 
solve some identified problems. 

Finally, build support on a daily basis. Once you have gained commitment, keep building 
upon it. 

At CertainTeed, a career executive on loan from the U.S. Department of Labor, served as 
the program advocate. With the help of a cadre of CertainTeed executives and a consultant, 
this advocate developed a plan and approach, sold the idea to top level CertainTeed 
executives, conducted upfront research, refined the plan, and presented the plan to the 
President and CEO of Saint-Gobain, CertainTeed' s parent company. The CEO accepted the 
plan and provided the necessary funding. His support for the program is reflected in his 
statement that "the job of business ... is to train educated employees so that they have the 
necessary skills to perform effectively." 

Certa : nTeed's training program also was supported by key executives at the plant level. The 
Oxford plant manager was extremely supportive of the training program. His willingness to 
balance the needs of the program and the demands he faced making and selling roofing 
shingles was invaluable. He realized that, when all was said and done, his production goals 
would be easier to reach if the skills of his workforce were enhanced. 

Since the introduction of the training program, a new plant manager has been assigned to 
Oxford. Because he was part of the leadership group that helped to design and develop the 
training program, he will continue to be a strong supporter and advocate of the program. 

Step 4: Build company ownership and worker involvement 

Once key executives are committed to the project, it is necessary to build support and 
ownership throughout the company. Sell the plant managers, plant leaders, and the local 
workforce on the project. Talk it up; inform, educate, and spark interest. 

At CertainTeed, plant level managers and supervisors embraced the project and the 
corporation's commitment, carrying out the actual day-to-day work of both the plant and the 
training program. The plant manager and his department heads were, by far, the most ardent 
and significant program advocates and supporters. They not only provided guidance and 
advice throughout the program, but also strong support - encouraging workers to attend, 
allowing workers to visit the training center for the first time "on-the-clock," serving as 
instructors, and generally "talking-up" the importance of participating in the program. 
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It is critical that employee involvement and support be institutionalized in the form of a 
steering committee established at the plant level to oversee, guide, and participate in program 
development. 



CertainTeed recognized the need for this support and established a local plant-level steering 
committee early in the program development process. This steering committee was made up 
of local plant management, as well as hourly production, maintenance, and warehouse 
employees. CertainTeed steering committee members were actively involved in all aspects of 
the training program, acting as a source of guidance and feedback on issues related to 
program design, data collection and analysis, curriculum, incentives, creation of a learning 
center, and selection and modification of courseware and instructional systems. 

In addition to these more technical tasks, the steering committee served as the mechanism for 
making company employees aware of the program and its goals, and generating widespread 
support for program activities. The committee members were educated about workplace 
literacy and its implications, the value of upgrading workers' skills, and the company's 
commitment to establishing an organizational culture that invests in and develops its human 
resources. Armed with this knowledge, committee members took on the task of creating 
enthusiasm for the workplace literacy program within the company. 
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CERTAINTEED'S STEERING COMMITTEE 
Purpose: 

To give advice and assistance on issues relating to 1) design of the training approach, 2) 
collection and analysis of data, 3) development of the training curriculum, 4) 
establishment of an incentive system, 5) creation of a learning center, and 6) selection 
and modification of courseware and instructional systems. 

Functions: 

• Help develop the general training approach. 

• Facilitate the "literacy job task analysis" and other data gathering efforts. 

• Review and validate the results of the data gathering and analyses. 

• Help develop the final training design structure and core curriculum. 

• Help determine the means for providing incentives. 

• Assist in the preparation of cost estimates and options. 

• Help decide where courseware can be purchased off the shelf and where custom 
courseware needs to be developed. 

• Help put training materials in the context of roofing products group jobs. 

• Help determine the requirements for and establish an in-plant learning center. 



Step 5: Recruit program partners 

Companies should not feel they need to develop and implement their precision strike training 
program on their own. There are many community resources that can be drawn on, 
including community colleges, vocational-technical institutions, and adult education 
programs; industry and trade associations; union training offices; and private consultants. 

The Presidential Exchange Program, a private sector/federal government exchange program 
sponsored by the White House, provided CertainTeed with an executive who was the catalyst 
and architect for its workplace literacy efforts. His services were augmented by those of a 
consultant with extensive experience in workplace literacy programs. 



Vance-Granville Community College provided CertainTeed with critical support and 
resources from the time of the program's inception, including over 3,000 hours of 
instruction, manuals and structured learning workbooks, development of a plant video, and 
administration of pre- and post-training tests. 



VANCE-GRANVILLE COMMUNITY COLLEGE 

The Vance-Granville Community College was a major partner in CertainTeed' s 
workplace literacy program. It administered pre- and post-tests evaluating workers' 
literacy skills, and provided extra instruction to employees reading at less than a fourth 
grade level. It offered grant money as seed money for either developing customized 
technical training courses or funding the evaluation effort. The college also agreed to 
develop job-specific training manuals for CertainTeed that could be used in conjunction 
with the company's on-the-job training component. Finally, the college assigned a part- 
time coordinator to act as liaison to CertainTeed and to direct the college's support of the 
project. 



The National Institute for Literacy (NIFL) funded a second component of CertainTeed 's 
training program, as well as an evaluation to determine individual learning gains and 
performance improvements. The NIFL grant also funded a series of "how to" workshops for 
Vance-Granville Community College workplace literacy instructors, to enhance their 
expertise in providing workplace literacy services to other industries in the Vance-Granville 
Community College service area. 

The first stop for most companies interested in pursuing a workplace literacy training 
program should be their local community college network. You should determine what 
resources the college can provide and solicit its support and commitment. This involves: 

• Connecting with your local community college and exploring your mutual interests, 
plans, problems, and options; 

• Seeing the college and involving it as a partner in the workplace literacy program, if 
possible; 

• Getting commitment and support in the form of funds, staff, and workplace literacy 
expertise; and 

• Appointing a liaison to negotiate, facilitate, and coordinate program efforts between 
your company and the college. 
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TASK 2: CREATE A WELL CONCEIVED PLAN 



A well conceived plan can be invaluable in designing and selling precision strike training. 
The foundation for this plan should be your company's: 

• Goals and objectives for the training program; 

• Guiding principles for its workplace literacy program; and 

• Program assumptions. 

From these, you should be able to establish a training approach and structure. 

Once you have crafted the plan, you should set it out in a training concept paper and use the 
paper to promote, design, and develop your precision strike training program. 

Step 1: Determine your program's goals and objectives 

It is critical that you determine your company's goals and objectives for its workplace 
literacy training program before proceeding far into the planning process. Company goals 
and objectives can be narrow or broad, and might include: 

• Ensuring that all existing employees acquire the skills to perform their current jobs; 

• Preparing new employees to work effectively on their assigned tasks; 

• Cross training employees so that they can substitute for other employees when 
needed; 

• Cross training employees so that they can work together as members of a team; 

• Providing employees with the skills to "trouble shoot" problems and make decisions 
on their own; and 

• Creating a continuous learning system that enables employees to upgrade their skills 
and prepare for higher- level jobs. 

The goal of CertainTeed's training program was to provide the company's roofing products 
group (RPG) with a fully integrated training and development delivery system that would 
allow workers to continuously upgrade their skills. 

Because CertainTeed's training program was a pilot project, the training objectives were 
focused on measuring the program's effectiveness, and included: 
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• Demonstrating a cost-effective means by which RPG workers can acquire basic and 
higher level functional literacy skills. 

• Demonstrating * cost-effective means to assure RPG workers are technically 
competent in the jobs they hold. 

• Demonstrating a cost-effective means to promote cross-training and succession 
planning for RPG workers. 

• Demonstrating that workplace literacy training will have a positive impact on such 
RPG "cost drivers" as process inefficiency, waste, delay, run rate, short term 
customer complaints, and consistency and accuracy of filler percentage and bundle 
weights. 

Step 2: Develop the principles that will shape your training program 

It is important that you establish a number of guiding principles to shape your workplace 
literacy program and ensure that it meets your company's needs. 

• Base your approach directly upon the business goals and objectives of your company. 

• Structure the training methodology to produce the best results in the shortest possible 
time. The focus, even in the midst of training, must remain on making a quality 
product. 

• Focus on skills that will transfer to the workplace and have a positive impact on such 
manufacturing cost drivers as process inefficiency, waste, delay, run rate, short term 
customer complaints, consistency, and accuracy. 

• Plan your approach to maximize learning gains. 

• Plan the training program so that it will be cost effective. Determine ahead of time 
the definition of "cost effective." 

• Focus your approach on teaching workforce literacy skills in the context of their job 
applications. 

• Train employees to completely master all of the skills they need to perform their jobs 
effectively. 

• Be sure training accommodates plant work schedules and shifts, and individual 
learning styles and preferences. 



12 



CEOTAINTEED'S PRECISION-STRIKE TRAINING PROGRAM 
Guiding Principles: 

1 . Program development and implementation should be guided by an orientation to 
the mission, goals, and objectives of the roofing products group. 

Although the general literacy needs of individual factory workers are important, 
the primary concern of the training program is to develop workforce literacy skills 
that will help the manufacturing company achieve its organizational goals and 
objectives. 

2. Training in workforce literacy skills must be provided in a functional context. 

This means that skills and knowledge are presented in a context that is meaningful 
to the person and his or her job. The more similar the workforce literacy skills 
training tasks are to actual job tasks, the easier it is for employees to learn the 
skills, the greater employees' motivation, and the more likely that the training will 
improve employees' work performance. 

3. Workforce literacy training programs must be arranged to maximize active 
learning time. 

Training time usually means time away from the production line. Therefore, 
whatever time is allowed for training must be well spent. Most individuals learn 
best when they are actively engaged in a learning task. Therefore, workforce 
literacy training programs should be designed so that trainees spend the greatest 
amount of time possible actively engaged in a learning task. 

4. Workforce literacy training programs should use a competency-based, mastery 
learning instructional approach wherever possible. 

This principle actually contains two ideas. Competency-based refers to basing 
trainees' learning goals on rhe trainees' actual job tasks, rather than on some 
external standard such as a grade school level or a percentile score. While 
mastery learning refers to requiring trainees to develop their skills to the level 
needed to perform all of their job tasks without mistakes, essentially requiring 
them to score 100 percent on all their learning tasks. 



Step 3: State your operating assumptions 

Every company will have its own operating assumptions that will guide its training 
development process. For CertainTeed, these assumptions defined workforce literacy and its 
relationship to company jobs. 
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CertainTeed first adopted the National Assessment of Educational Progress* 1986 definition 
of workforce literacy: 

Workforce literacy means the full range of reading, writing, oral 
computational analytical and reasoning skills that enable an individual to use 
print ed and written information; to function fully in society and in his or her 
job; to achieve personal goals; to continue learning and develop knowledge; 
and to reach his or her potential 

CertainTeed then assumed that: 

• Line jobs require definable levels of proficiency in basic literacy, functional literacy, 
and technical competence. 

• A base level of necessary "basic literacy" skills exists for all line jobs and can be 
determined; and 

• Line jobs can be grouped by levels of training needs. 
Step 4: Formulate your training approach 

A company's training approach should outline its "operating rules" for the program. 

CertainTeed established that training would be voluntary and non-participation would not 
adversely affect workers; incentives would be provided to encourage employees to attend 
training; all training-related records would be confidential; and all workers would have equal 
access to training. 

Further, existing employees would be required to demonstrate mastery of fundamental basic 
literacy skills as a prerequisite to entering the CertainTeed training program, and an 
individualized literacy training program would be established with local schools for 
CertainTeed employees lacking these fundamental skills. All new hires < /ould be required to 
possess a foundation level of basic literacy skills. 

Roofing products group jobs would be grouped into levels around similar training needs; 
stand-alone, open-entry, open-exit training curricula designed to meet these needs would be 
developed at each level; and employees would be allowed to enroll in courses at the training 
level of their jobs or the next higher training level, as long as they completed courses at the 
level of their jobs first. 

Finally, orientation training would be developed and given to all new hires. 
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THE CERTAINTEED TRAINING CONCEFr 



CertainTeed's training design was structured around the assumption that there was a 
fundamental level of basic literacy skills that roofing products group workers needed in 
order to be effective on the job. The design further assumed that: 

• Whether an employee had this fundamental level of basic skills could be 
determined by an assessment tool; 

• The basic literacy skills would provide a foundation around which subsequent 
training could be designed and developed; 

• Roofing products group production line jobs could be grouped into training levels 
that required increasingly higher levels of literacy skills; and 

• Training could developed to meet the skill needs. 

From these assumptions, CertainTeed began to establish its training approach: 

• Training levels would be established that loosely tracked the hierarchy of existing 
roofing products group production line jobs; 

• Each training level would integrate the teaching of the literacy and technical skills 
needed to perform the jobs well, and would allow for personal growth; 

• Training would consist of a combination of multi-media, self-instruction modules 
in a plant-based learning center, on-the-job training, job rotation assignments, and 
classroom instruction; 

• As a prerequisite for acceptance into the training program, existing employees 
would be required to demonstrate that they possessed the fundamental level of 
basic literacy skills; 

• Those who .did not possess basic literacy skills at the fundamental level would be 
encouraged and provided the opportunity to obtain the skills, but they would not 
be required to undertake the additional training; 

• Employees could choose to enroll in training at the level in which their current job 
fell and one level above that; 

• An employee would be required to complete successfully the training curriculum 
for the training level in which that employee's job fell before being allowed to 
enroll in the training at the next higher level; 



THE CERTAINTEED TRAINING CONCEPT (continued) 

• All new employees would be given a one-week orientation training upon hire and 
before beginning work; 

• Orientation training would focus on ensuring that new employees understood 

- the roofing products group's (RPG) quality and safety goals, 

- the overall RPG production line process, 

- CertainTeed, plant, and RPG policies, and 

- what was expected of them and what opportunities were available to them; and 

• New employees would be allowed to enroll in the training program for their level 
immediately after completing the orientation training. 

To ensure a future flow of job applicants that possessed the fundamental level of basic 
literacy skills required by the roofing products group, CertainTeed established cooperative 
relationships with the schools, community colleges, Job Training Partnership Act service 
providers, and local job service offices in each plant's recruiting area, and communicated 
the roofing products group's basic literacy requirements to them. 



Step 5: Create your training structure 

The first step to creating a training structure is to determine the scope of your initiative - 
bounded by the level of resources you are willing to devote to the effort. Once your 
company has set this boundary, you are ready to create your training structure - where, 
when, how and for whom training will be provided. 

CertainTeed decided to focus its training program at the Oxford plant, one of three plants 
that manufactures CertainTeed asphalt roofing shingles. The Oxford plant is the most 
technologically modern of the three shingle plants. CertainTeed has been using the Oxford 
plant to introduce new production processes, new products, new machines, and new 
technologies. The fast paced changes occurring in this plant have placed a tremendous 
responsibility on management and workers to perform. The workforce of 200 includes 
approximately 170 production, warehouse, and maintenance workers. CertainTeed pays 
excellent wages for the area. Turnover is minimal, providing a stable workforce. The plant 
is non-unionized. Worker involvement in all aspects of plant operations is a cornerstone of 
the working environment. The plant manager is a strong program advocate and is committed 
to the use of training to build a quality workforce. 
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CertainTeed's training program was to be focused on 17 key manufacturing jobs: 



Utility Operator 
Grand Manor Operator 
Hallmark Operator 
Loader (Warehouse) 
Supply/Checker 
Supply/Service (Membrane) 
Wrapper/Palletizer Operator 
Dry End Operator 
Surface Operator 



Cutter/Winder Operator 
Service/Supply (Yard) 
Coater Operator 



Top Slate Operator 
Stillyard Operator 
Inspector 



Maintenance Electrician 
Maintenance Machinist 



Initially, curricula would be developed for clusters of similar jobs. Later, it would be 
developed for each individual job. Trainees would be assessed before they began training, 
and would only have to study the skills in their training level or job curriculum for which the 
assessment showed a deficiency. 

CertainTeed decided to establish a learning center in the Oxford plant which would include 
computer work stations and computer-aided instruction, video and audio playback units, 
programmed learning and self-instruction courseware, and training reference and resource 
materials. The program's self-instruction modules would be supplemented with appropriate 
classroom training, some of which, at the higher training levels, would be provided by 
corporate headquarters. The plant's training supervisor would administer the program, and 
the center would be staffed for about 40 hours per week. CertainTeed's program gave 
employees some release time to use the learning center, but for the most part, participation 
was to be 50 percent on employee time and 50 percent on company time. Finally, the 
company decided to expand its community "adopt-a-school" program in order to ensure a 
supply of well-prepared workers in the future. 
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TASK 3: CONDUCT UPFRONT RESEARCH AND ANALYSIS 



Extensive up-front research and assessment is required before a company's workplace 
literacy program can be developed. You need to define the gap between the current skill 
levels of the workforce and the levels needed for peak performance in order to develop 
curricula that will fill that gap. Defining the gap and preparing for curricula development 
involves: 

• Determining the literacy skills needed to effectively carry out targeted jobs in your 
company; 

• Developing skill profiles for each job or group of jobs; 

• Assessing the skill levels of current employees; and 

• Determining employees' learning needs, preferences, and interests. 
Step 1: Determine the literacy skills needed for targeted jobs 

Companies should undertake a number of different procedures to determine the literacy skills 
required for the jobs targeted for training. These procedures, usually referred to as literacy 
task analyses, include: 

• Conducting panel discussions with job experts; 

• Observing workers at their jobs; 

• Interviewing workers about their jobs; and 

• Reviewing and analyzing written material used on the job. 

These investigations should focus on identifying the literacy skill applications embedded in 
key job tasks. In the end, you should be able to develop literacy task statements that 
describe what employees must be able to do, under what conditions, and how well. These 
literacy task statements will serve as the basis for the development of curriculum, assessment 
tools, and job performance and productivity measures. 

After the initial literacy skill data were gathered at CertainTeed's Oxford plant, the results 
were compiled by job and documented in a format that listed the job task and accompanying 
literacy requirements side-by-side. Members of the literacy analysis team then traveled to 
the Avery and Shakopee plants and conducted mini-literacy skill analyses to verify the 
information gathered in Oxford. Any differences in job tasks and literacy applications at the 
verification sites were noted for use later should the workplace literacy program be 
expanded. 
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Panel Discussions. Panels of experts can be convened to generate data about the jobs 
targeted for training. The panelists should comprise key hourly staff identified as being 
expert in the jobs being analyzed, first line supervisors, and management officials. These 
panels should: 

• Determine key tasks for each job; 

• Rank order the tasks from most to least important; 

• Identify the evaluation criteria against which successful completion of these tasks 
would be measured; and 

• Determine, for each job, a set of key attributes and/or capabilities that employees 
should bring to the jobs. 

At CertainTeed, a total of seven panels were convened to generate data on seventeen 
different roofing products group jobs. Each panel required about two hours per job to 
complete the analysis. The panels were facilitated by a two-person team composed of 
CertainTeed project staff and/or the consultant. 

The success of CertainTeed' s panel component was due to a combination of motivated and 
knowledgeable panelists and the technique used during the process — the nominal group 
technique. 

The nominal group technique was selected as the procedure of choice for the panel 
component because it is a structured activity that provides an orderly procedure for 
systematically obtaining qualitative information from a group, and assessing priorities on that 
information. Individuals work in the presence of others but don't interact verbally except at 
specified times and in limited ways. The nominal group technique process used with the 
panels involved four stages: 

1) Silent generation of ideas and responses in writing; 

2) Round-robin feedback from panel members to record each idea and response in a 
terse phrase on a flip chart; 

3) Discussion of each recorded idea and response for clarification and evaluation; and 

4) A group rank ordering of the ideas and responses based on a tally of individuals' own 
priority rankings. 

The job task information generated during the panel component was recorded on flip charts 
and then input into a computer. The computer print-outs were later used during the 
observation and interview components of the analysis. 
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Observations. The information obtained through panel discussions can be augmented by 
structured observations of the jobs targeted for training. Through these observations, you 
can gather and record information about job tasks and their associated skill requirements. 
During the observations, you should identify and record: 

• The primary and the most frequent tasks associated with each job; 

• The action steps taken in carrying out these tasks; and 

• The skill requirements (reading, writing, verbal communication, computation) 
associated with completing the tasks and steps. 

Fifteen jobs were observed at CertainTeed. Observations occurred over a three to four hour 
continuous period to ensure that most of the tasks required by the job were observed. The 
observer asked questions as needed to clarify what was being done and what was involved in 
doing it. The person being observed was asked to identify any key tasks associated with his 
or her job that were not being observed directly. 

The guidelines used for the worker observations at CertainTeed were: 

• Try to identify recurring job tasks, and distinguish these tasks from their component 
action steps; 

• If possible, determine the purpose of the task; 

• Record each time a worker reads, writes, does an arithmetic calculation, or uses 
mathematics principles; 

• Note the setting in which these basic skill activities take place; 

• Note the materials used by the employee to perform the task involving basic skill 
activities; 

• If in doubt about a basic skill activity or its purpose, ASK QUESTIONS! 

The results of each observation were documented and filed with the results of the panel 
component. 

Interviews. Interviews with individuals performing each job can be conducted to verify and 
elaborate on the information gathered during the panel and observation components. A 
critical aspect of the interview component is to identify and document the thinking processes 
that the workers use as they perform the literacy components of their key job tasks. 
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For each of the fifteen jobs analyzed in the panel and observation components at 
CertainTeed, interviews were conducted with two additional individuals whose job 
performance was not observed. 

Interview questions were designed to elicit such information as: 

• The type of written material used on the job; 

• The use made of written material used on the job; 

• The frequency with which written material is used on the job; and 

• The kind of writing done on the job. 

Analysis of Written Material. A key part of determining the literacy skills required of 
particular jobs is conducting a readability assessment to determine the difficulty level of the 
job-related reading material the workers must use. These readability assessments are 
important steps in understanding the gap that may exist between the literacy levels of the 
workers and what is required of them on the job. 

A wide variety of work-related materials were assessed at CertainTeed. Some of the 
material provided general information about CertainTeed rules, regulations, benefits, 
policies, and procedures. Most of the material specifically related to performance on the job. 

In collaboration with CertainTeed staff at corporate headquarters and staff at the Oxford, 
Avery, and Shakopee plants, approximately 300 samples of workplace reading materials were 
selected for study. 

Eight categories of material were tested: 

• Policies and general information for all employees 

• Job Safety Analysis Training Guides 

• Job Safety Analysis Worksheets 

• Job instructions/procedures/product specifications 

• Job checklists and check sheets 

• Forms and reports 

• Letters, memos, and minutes from various meetings 

• Job specific technical training manuals 

CertainTeed used the FORECAST (an acronym which comes from the names of the 
researchers who developed it — Fox, Ford, Cay lor and Sticht) readability formula to 
determine the reading difficulty level of workplace materials. The results of the FORECAST 
readability formula are expressed in terms of the estimated reading grade level required. 
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FORECAST has a high statistical correlation to other readability formulas. It was selected 
for use at CertainTeed because it is more accurate with workplace materials than other 
formulas in general use because it is designed to compensate for the large number of 
multiple-syllable words commonly found in technical materials in the workplace. (An 
abundance of technical words can skew the results of readability formulas and tests.) 
FORECAST is also relatively quick and easy to use. 

The 300 samples of workplace reading material, consisting of 150 word passages each, 
produced 450,000 words which were counted and analyzed for reading level difficulty. 

This huge volume of data was compiled and organized in the following way: 

• Aggregate data, with mean readability scores; and 

• Readability level of workplace reading materials by 

- subject category (charts, checklists, etc.), 

- individual samples (e.g., all Job Safety Analyses), and 

- job function and title (hallmark operator, etc.). 

The range of difficulty for all reading materials tested was from grades 6+ to 15 + . 
Step 2: Develop skill profiles for each job or group of jobs 

Once you determine the literacy skills required for each job, you need to create skill profiles 
that clearly spell out the competencies employees must attain in order to perform adequately 
on the job. These skill profiles will serve as the base for your precision strike training 
program, defining both curriculum and assessment tooL. 

Initially, CertainTeed clustered the various roofing products group jobs into training levels 
with similar skill requirements so that it could create a simpler and more coherent training 
program. These training levels allowed CertainTeed to create skill profiles for groups of 
jobs, rather than for each job. Later, CertainTeed decided to develop more job-specific 
curricula, and needed to generate skill profiles for individual jobs. 

While grouping jobs into training levels is not required, you may want to consider it as a 
way to organize your company's training program as well. 

To group the jobs, CertainTeed rated the job tasks and accompanying literacy skill 
applications for the jobs against 16 standards: 

• Oral comprehension 

• Written comprehension 
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• Oral expression 

• Fluency of ideas 

• Originality 

• Problem sensitivity 

• Mathematics reasoning 

• Number facility 

• Deductive reasoning 

• Inductive reasoning 

• Information ordering 

• Flexibility of closure 

• Spatial orientation 

• Visualization 

• Control precision 

• Rate control 



Having done this, CertainTeed was able to create five training levels with similar literacy 
requirements. 



CERTAINTEED'S TRAINING LEVELS 




Training Level I 


Training Level IV 


All New Hires 
Laborers 

Third and Fourth Hands 
Clean-up 
Utility Operators 
Grand Manor Operators 


Saturator/Coater Operators 
Granule/Top Slate Operators 
Boiler/Stillyard Operators 
Inspectors 
Lead Relief 
General Operators 


Training Level II 


Training Level V 


Hallmark Operators 
Loaders (Warehouse) 
Supply/Checkers 
Supply/Service (Membrane) 
Unloaders 


Maintenance Electricians 
Maintenance Machinists 
Maintenance Mechanics 


Training Levei III 




Wrapper/Palletizer Operators 
Dry End (Felt/Membrane) Operators 
Surface Operators 
Cutter/Winder Operators 
Service Supply (Yard) 
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Once the jobs were grouped into training levels, CertainTeed developed skill profiles for 
each training level. To do this, CertainTeed first defined five skill areas - basic skills, 
functional skills, technical skills, work maturity skills, and physical requirements - and the 
specific skills contained in each area: 

Basic Skills 

Learning-to-learn skills 
Reading skills 
Writing skills 
Computation skills 

Functional Skills 

Communication skills 

Group effectiveness skills 

Problem solving and analytical thinking skills 

Computer literacy skills 

Technical Skills 

Use of tools and equipment 
Knowledge of machines 
Knowledge of instrumentation 

Work Maturity Skills 

Punctuality 

Readiness to work at scheduled times 

Willingness to accept work assignments without complaining 

Appropriateness of grooming and dress 

Acceptance of supervision and guidance 

Interpersonal relations 

Ability to follow directions 

Timely and effective completion of tasks 

Ability to carry out work duties independently as required 

Cooperation with other employees 

Acceptance of constructive criticism 

Show of interest and enthusiasm for the job 

Physical Requirements 

Amount of force required to perform the job and the frequency of that requirement 
Requirements to climb, balance, stoop, kneel, crouch, crawl, and reach 
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Requirements to identify and distinguish colors 
Requirements to tolerate noise, vibration, and weather 
Ability to visually identifying defects in objects 
Ability to perceive the size, shape, and texture of objects 
Ability to discriminate differences in sounds 
Ability to judge spatial and temporal relationships 
Ability to identify and distinguish colors 

CertainTeed then used these specific skills and skill areas to develop distinct skill profiles for 
each training level. The skill profile for Training Level V serves as an example: 



CERTAINTEED TRAINING LEVEL V: SKILL PROFILE 

Jobs Covered: Maintenance Electrician, Maintenance Machinist, Maintenance Mechanic. 

Key Characteristics: These jobs are characterized by high level repair skills that involve 
reading and comprehending complex diagrams, schematics, sketches, blueprints, 
operation manuals, and manufacturers' specifications. Identifying and repairing 
malfunctions is a critical aspect of these jobs. 

These jobs require the use of such precision test and measuring instruments as ammeters, 
oscilloscopes, test lamps, micrometers, and calipers. Analytical and problem-solving 
skills are critical. 

Results of Readability Analysis: A readability assessment of job-related materials for 
jobs at this level shows reading requirements ranging from a low of grade 7 + to a high 
of grade 13 + . Materials reviewed included task-specific Job Safety Analyses (which 
ranged from grades 7 to 11+), and job-specific instructions and procedures (which 
ranged from grades 10 to 13+). 

BASIC SKILLS 

Reading Skills 

• Recognize common words and meanings, task-related words with technical 
meanings, and meanings of common abbreviations and acronyms. 

• Recognize job-specific words with technical meanings and job-specific 
abbreviations and acronyms. 

• Follow sequential directions and/or illustrations to complete a task. 
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CERTAINTEED TRAINING LEVEL V: SKILL PROFILE (continued) 

• Use a completed form to locate information to complete a task. 

• Classify or match objects by color, size, or significant marking. 

• Locate pages, titles, paragraphs, figures, or charts needed to answer questions or 
solve problems. 

• Recognize computer-related jargon and acronyms. 

• Use a table of contents, index, appendix, and glossary. 

• Identify factual details and specifications within text to complete a task. 

• Locate individual specifications or facts in text. 

• Read and interpret flow charts and schematics that sequence events or structure 
processes. 

• Use flow charts to solve problems. 

• Determine the presence of a defect or extent of damage to a product or raw 
materials. 

• Cross reference within and across source materials to select information to 
perform a routine. 

• Read two or more column charts to obtain information. 

• Locate chart information at the intersection of rows and columns. 

• Use common knowledge to operate safely. 

• Apply preventive measures prior to undertaking a task to minimize problems. 

• Read and interpret numbers and symbols from gauges, and display screens such as 
those measuring temperature (in degrees centigrade or fahrenheit), pressure (in 
PSI), or speed (in RPM). 

• Enter information into a computerized system using a keyboard or keypad. 
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CERTAINTEED TRAINING LEVEL V: SKILL PROFILE (continued) 

Combine information from multiple sources that contribute to the completion of a 
task. 

Apply information to locate malfunctions or decide upon a course of action. 
Interpret codes and symbols. 

Isolate problems in schematics, tracing the cause of the problem. 
Read and interpret blueprints and blueprint symbols. 
Writing Skills 

Enter information into a form (numbers, weight, job-related words). 

Record essential information in phrases or simple sentences accurately and 
precisely. 

Write key technical words accurately on forms. 

Transfer numbers, codes, dates, and numbers from equipment, products, or other 
written sources into appropriate sections of forms. 

Spell task-related and job-related technical words, abbreviations, and acronyms 
correctly. 

Outline a process by arranging key events sequentially into a step-by-step 
procedure. 

Generate a written communication according to a specific format. 

Write brief justifications for actions taken and provide good reasons for rejecting 
alternative solutions. 

Write a report including necessary support documentation or classification. 
Summarize events in an accurate and complete manner. 
Computation Skills 

• Read, write, and count single and multiple digit whole numbers. 
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CERTAINTEED TRAINING LEVEL V: SKILL PROFILE (continued) 

• Add, subtract, multiply, and divide single and multiple digit numbers. 

• Read and write common fractions. 

• Read and write decimals in one or more places. 

• Read and write percentages. 

• Read numbers or symbols from a weight or measuring scale. 

• Use a measuring rule to determine an object's physical dimensions in standard 
units. 

• Read and tell time. 

• Use addition, subtraction, multiplication, and division to solve problems with 
single and multiple digit whole numbers. 

• Round off single and multiple digit numbers, fractions, and decimals. 

• Solve problems with common fractions and decimals. 

• Use a calculator to perform basic arithmetic operations to solve a problem. 

• Solve math problems by selecting and using the correct order of operations. 

• Perform calculations quickly. 

• Compute averages, percentages, means, standard deviations, and ranges. 

• Read and construct X-bar and R-bar charts. 
FUNCTIONAL SKILLS 

Communication Skills 

• Follow spoken directions. 

• Extract information from an extended message. 

• Recognize non-verbal information (e.g., tone, gesture, attentiveness). 
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CERTAINTEED TRAINING LEVEL V: SKILL PROFILE (continued) 

State information concisely and efficiently. 
Check that messages have been understood. 
Listen for content. 

Use a question and answer format to establish and meet a need. 

Recognize communication styles by how they are manifested, and understand how 
the styles affect the message being communicated. 

Recognize shifts in communication purpose. 

Determine underlying assumptions and biases. 

Evaluate messages in terms of credibility, usefulness, and appropriateness. 
Utilize listening skills. 
Group Effectiveness Skills 

Contribute information in keeping with the topic. 
Express opinions and judgements (using feelings and intuition). 
Function in a group support role. 

Support a group process by applying knowledge of group dynamics. 

Manage conflict on the job by using conflict resolution skills. 

Serve as a fully functioning member of a process improvement team using group 
task and maintenance skills. 

Follow a line of thought as it develops among several speakers/participants of a 
group to keep the group focused. 

Place individual processes or information in the "big picture." 
Exercise effective leadership when in groups or teams. 
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CERTAINTEED TRAINING LEVEL V: SKILL PROFILE (continued) 
Problem-Solving and Analytical Thinking Skills 

• Recognize task-specific problems and take appropriate action. 

• Analyze a problem to determine its cause and suggest alternative courses of 
action. 

• Develop and evaluate ideas deductively (apply a general principle to a specific 
situation by using analogy or generalization to form a hypothesis, then exploring 
and testing its implications, and finally recommending or not recommending its 
adoption). 

• Utilize Pareto (80/20 rule) and Fishbone (ishekawa chart) methods of analysis. 

• Disaggregate data from a conceptual model, flow chart, or schematic to 
constituent parts. 

« Develop and evaluate solutions to problems. 

• Make inferences from empirical data. 

• Read and construct process flow charts. 

• Identify ways to improve the operation of various manufacturing processes through 
improvements to machinery and electrical devices. 

Computer Literacy Skills 

• Read and comprehend screen displays and distinguish among key variables in that 
display. 

• Enter data using a computer keypad or a punch pad. 

• Using defined function keys and written procedures, move between and among 
screen displays to find appropriate data. 

TECHNICAL SKILLS 

Use of Tools and Equipment 

• Measure materials with a tape measure to within 1/16 of an inch. 
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CERTAESTEED TRAINING LEVEL V: SKILL PROFILE (continued) 

• Use hand tools such as a wrench, screwdriver, or scraper to make minor 
adjustments to equipment in order to perform preventive maintenance or clean up. 

• Operate a forklift. 

• Cut or trim material using a knife or cutting tool. 

• Use hand tools and power tools and precision measuring and testing instruments 
including: 

- calipers 

- micrometers 

- ammeters 

- oscilloscopes 

- test lamps 

- levels 

- plumb bobs 

- straight edges 

• Operate a genie boom. 
Knowledge of Machines 

• Use hands or hand tools to clean, remove, or carry objects or materials. 

• Observe machinery or mechanical objects and listen to sounds to locate causes of 
trouble. 

• Dismantle machinery and electrical equipment. 

• Adjust functional parts of devices, machinery, and control instruments. 

• Replace and repair defective parts, wiring, switches, burnt out elements, or fuses. 
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CERTAINTEED TRAINING LEVEL V: SKILL PROFILE (continued) 

• Start devices, machinery, and electrical devices to test performance. 

• Set up and operate lathes, drill presses, grinders, and other metal working tools to 
make and repair parts. 

• Locate and determine electrical malfunctions using testing instruments. 
WORK MATURITY SKILLS 

• Accept criticism without showing anger, blaming others, or making excuses. 

• Do not complain when asked to complete a task, and perform tasks as requested. 

• Show interest and enthusiasm in the job. 

• Follow both verbal directions and written plant policies. 

• Ask for more information when needed. 

• Complete job tasks in a timely manner. 

• Report to work as scheduled and give reasonable notice if you must be absent; 
start and stop work at the scheduled time; leave and return from all breaks at the 
correct time. 

• Use tools and equipment safely and correctly. 

• Keep work area clean and safe. 

• Cooperate with other employees. 

• Respect the property of others. 

• Know duties and responsibilities and work without close supervision. 

• Work effectively as pan of a team, often assuming a leadership role. 

• Show complete independence in carrying out most work duties, requiring 
supervision only to assist in solving the most difficult problems. 

• Assist other employees with their jobs when workload allows. 
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CERTAINTEED TRAINING LEVEL V: SKILL PROFILE (continued) 
PHYSICAL REQUIREMENTS 

• Exert up to 100 pounds of force occasionally, and 30 to 50 pounds of force 
frequently. 

• Climb, balance, stoop, kneel, crouch, crawl, and reach. 

• Identify and distinguish colors. 

• Tolerate noise, vibration, and some limited exposure to weather. 

• Visually identify defects in objects. 

• Perceive attributes of objects, such as size, shape, and texture by touching with 
fingers. 

• Discriminate differences in sounds. 

• Judge spatial and temporal relationships. 



Step 3: Assess the workplace literacy skills of current employees 

In order to design appropriate workplace literacy curricula, you not only need to know the 
skills required of particular jobs, but also the skills employees currently have. 

CertainTeed used two different tests to assess the workforce literacy skills of job 
incumbents — a Cloze test and the Test of Adult Basic Education (the TABE test). 

Cloze Test. The Cloze Test is an inexpensive, easy to administer measure of the ability of a 
reader to understand the sentences of a passage at a literal or factual level. It is often used 
in workforce literacy training to assess the reading level of job incumbents. At CertainTeed, 
the Cloze Test was administered as part of the Employee Learning Preference Survey, and 
voluntarily completed by 418 workers from the Oxford, Avery, and Shakopee plants. 

A Cloze test can be constructed at any reading level desired. At CertainTeed, a reading 
passage that was assessed to be at the 9+ grade reading level was selected. This level is 
slightly below the mean difficulty level of all job-related material assessed at CertainTeed. 
However, it is at the upper end of what is commonly referred to as the mid-level range of 
literacy. Individuals who can read and comprehend material at this level are prepared to be 
trained for higher levels if needed. 



Cloze Tests are constructed by selecting a reading passage, conducting a readability 
assessment to determine its difficulty level, and deleting words from the passage for the 
learner to fill in. 

Tests are scored by simply counting the number and calculating the percentage of correct 
responses. CertainTeed's scale for interpreting Cloze text results was: 

• A 70 percent or higher accuracy rate in responses on a Cloze passage written at the 
9+ level indicated an employee with advanced level literacy skills who could easily 
comprehend materials at this level independently , or without assistance of an 
instructor. 

• A 43 to 65 percent accuracy rate in responses indicated an employee with 
intermediate-level literacy skills who could successfully comprehend materials at this 
level with some instructor assistance. 

• Scores of below 43 percent accuracy rate in responses indicated an employee who 
would have difficulty comprehending 9 + materials and would become frustrated 
without considerable help, support, and/or prerequisite remedial basic skill 
instruction. 

The volume of data generated by the Cloze Test was extensive. The 418 scores of the 
workers who voluntarily completed the Cloze Test were compiled by: 

• Individual worker 

• Job title 

• Work areas (production, stillyard, felt mill, maintenance, warehouse, office) 

• Plant (Oxford, Avery, and Shakopee) 

• Total - all plants 

In addition, results of the Cloze Test were correlated with various responses generated from 
the Employee Learning Preference Survey (discussed in the next step). 

TABE Test. The Test of Adult Basic Education (TABE) is a well accepted, commonly used 
assessment instrument designed to assess the basic skills of adults. The TABE was 
administered to 167 workers from the Oxford plant and scored free-of-charge by the Vance- 
Granville Community College. 

The purpose of the TABE is to provide an indepth assessment of the basic skill levels of 
adults. The TABE measures the following basic skill areas: 

• Reading vocabulary 

• Reading comprehension 

• Language mechanics 
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Language expression 

Spelling 

Computation 

Mathematical concepts and application 



The first step in the TABE testing process was the administration of a locator test. This test 
determined what difficulty level of the complete test should be administered to each 
individual. Once this was accomplished, the complete test was administered. 

TABE results were used in the evaluation model as a method of measuring individual and 
aggregate gains in learning. The initial results were compared with results after completion 
of the 18-month pilot training program. 

Standardized tests such as the TABE are commonly used by community colleges across the 
country, and provide a good assessment of an individual's general basic skill abilities. In 
addition, standardized tests can be used as program pre- and post-assessments to measure 
gains in general reading abilities. The results are depicted in grade levels which are easily 
understood by program participants as well as those who are funding or sponsoring the 



There are several disadvantages to using only standardized tests in a workplace literacy 
program. Standardized tests measure general basic skill abilities and not the specialized 
literacy skills required in the workplace. A learner could move from the 5th grade reading 
level to the 8th grade reading level, and therefore be better able to read newspapers and 
magazines, but not be able to easily transfer the new skills to reading a statistical process 
control chart, a technical training manual, or a Job Safety Analysis Training Guide. In 
addition, gains individuals make in their ability to read work-related materials may be only 
partially reflected in a standardized test. 

Most workplace literacy experts and practitioners recognize these limitations in the use of 
standardized tests such as the TABE. Standardized tests can provide useful information. 
However, they should never be used alone but should be used as a part of a mix of 
assessments that are custom designed based on workplace materials and activities. 

In general, standardized tests such as the TABE will not meet the needs of companies 
planning to implement precision strike workplace literacy training. Standardized tests usually 
measure the ability to use basic skills in a schoolroom-like environment and require an 
average of one to three hours to administer. The Cloze test is a more practical and equally 
effective alternative. While Cloze test results correlate highly with the results of 
standardized tests, they measure the ability of a worker to understand work related materials 
written at a particular level of difficulty. In addition they require only 10 or 15 minutes to 
deliver. 
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Since CertainTeed developed its training program, a new literacy assessment tool — the Tests 
of Applied Literacy Skills (TALS) - has become available. The TALS are designed to assist 
program providers and staff in estimating the literacy proficiencies of individuals applying 
for or receiving services; to assist in placing individuals into appropriate, existing programs; 
and to provide a means for assessing learner progress over time. 

The TALS evaluate individuals' literacy skills on three scales - prose, document, and 
quantitative — and determine their placement in one of five levels for each of the three 
scales. The literacy levels on each scale do not say what specific level of prose, document, 
or quantitative skill is required to obtain, hold, or advance in a particular occupation. 
However, to the extent that the tasks that an individual at a given level can do correspond to 
those in your workplace, using these scales enables you to assess the needs of the workers in 
your literacy program, and then prescribe the appropriate curriculum. 

Step 4: Determine employees' learning needs, preferences, and interests 

In order to maximize the likelihood that employees will participate in and learn from the 
training programs, the design and development of precision strike training should be based 
on employees' views of their learning needs, preferences, and interests. To determine these, 
you need to conduct an employee survey. 

The purpose of CertainTeed' s survey was to obtain information from the roofing products 
group workers themselves for use in planning training courses and programs that met the 
workers' learning style preferences, addressed their interests, and met CertainTeed 's needs. 

The Employee Learning Preference Survey was administered on a voluntary basis to roofing 
products group workers in the Oxford, Avery, and Shakopee plants. Four hundred sixty-one 
workers — an overwhelming response — completed the survey. 

The survey queried workers in the following areas: 

• Method of training for current job, if any; 

• Prior schooling/training; 

• Courses desired; 

• Type of training needed in speaking, reading, writing, math, and measuring; 

• Most and least preferred learning methods and techniques; 

• Preferences for study/ learning setting, supervision, and directions; 

• Frequency of reading for pleasure and on-the-job; and 

• Ability to ask for help. 
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The survey also captured basic information about each respondent: age, educational level, 
title of current job, and length of time in job, work area, and plant location. 

The Employee Learning Preference Survey presented each worker with a maximum of 195 
possible choices or responses. Consequently, the volume of data generated by the survey 
was substantial - a possible 89,995 responses for the 461 workers voluntarily completing the 
questionnaire. However, since the survey was structured to capture only preferences among 
a list of choices, the actual number of responses generated and recorded was in the range of 
40,000 for all respondents. 

Data from each of the 13 questions and their subparts was recorded and studied on several 
levels: 

• By individual worker and job title; 

• By work areas for each of the three plants; 

• By each plant total; and 

• By overall totals for all three plants. 

Responses from the survey were compiled into 447 charts. 
Some key findings for all three plants (461 respondents) are: 

• Training for Current Job. Seventy percent of the respondents were either trained 
for their current job by the previous job holder or received no training. 

• Training Courses Desired: 

- Computer programming - 48% 

- electricity -37% 

- drafting and blueprints - 31% 

- electronics - 30% 

• Requests for Help or Training: 

- Giving reports to larger groups - 48% 

- Reading blueprints and schematics - 52% 

- Spelling and punctuation - 20% 

- Calculating mean and standard deviation - 49% 

- Instruments: 

- Vernier Caliper - 50% 

- Micrometer - 46% 
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- Volt/Qhm Meter - 44% 

- Go-no-go meter - 42% 

• Most Preferred Learning Method: 

- Working with hands - 75% 

- Reading and practicing - 52% 

- Demonstration - 44% 

• Least Preferred Learning Methods: 

- Deadlines and pressure - 41% 

- Lectures and notes - 31% 

- Speech - 27% 

• Pleasure Reading Frequency: 

- Every day - 38% 

- 2-3 times per week - 36% 

- Never - 2% 

• On-the-Job Reading Frequency: 

- Every day - 54% 

- 2-3 times per week - 14% 

- Once a week - 5% 

- Never - 8% 
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II. CURRICULUM 



TASK 1: DETERMINE WHETHER YOU CAN USE EXISTING TRAINING 
RESOURCES 

In many cases, existing training resources (systems, materials, curricula, technology, and 
delivery systems) can be used to meet your company's training needs. Before developing 
new materials, you should identify and evaluate existing resources and compare them to your 
particular training needs and requirements. To do this you should: 

• Determine your current and future training needs; 

• Create guidelines for selecting training resources; 

• Review existing training resources; and 

• Select those training resources that best meet your needs. 
Step 1: Determine your current and future training needs 

If you have done a good job laying the groundwork for your workplace literacy program, 
determining your company's current and future training needs should be easy. These 
aining needs will be defined by: 

Your company's program goals and objectives; 

The principles shaping your company's training program; 

Your company's operating assumptions; 

The training approach selected by your company; 

The training structure your company designed; 

The literacy skills needed for targeted jobs; 

The workplace literacy skills of current employees; and 

Employees' learning needs, preferences, and interests. 
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Step 2: Create guidelines for selecting training resources 



Before reviewing existing training resources, you will need to create a set of guidelines - 
customized to your own training needs - for judging the materials. A first step in creating 
these guidelines is to determine the training methodologies - such as computer-based 
instruction, interactive video instruction, classroom training, structured on the-job training, 
workbooks, or other media - most appropriate for meeting your company's training needs. 

After determining the most appropriate training methodologies for its needs, CertainTeed 
drew from various sources of technical expertise, such as "Characteristics of a Good 
Instructional System," "Special Features of Computer Assisted Instruction," and "Comparing 
Individual and Comprehensive Learning Software Programs" to identify its set of guidelines. 
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CHARACTERISTICS OF A GOOD INSTRUCTIONAL SYSTEM 

The curriculum and the instructional program: 

• Are based on competencies required for a specific job; 

• Inform the trainee what to expect and describe both the system's and the trainee's 
responsibilities; 

• Place the training within the appropriate occupational context, i.e. provide the 
trainee with a general description of the occupational area, a description of the 
job, the conditions under which the job is generally performed, and the generally 
accepted standards for the job's performance; 

• Are made up of carefully engineered learning experiences designed to help 
trainees acquire the specified competencies; 

• Allow the trainee to control the pace, sequence, and strategy of the learning; 

• Demonstrate or describe the skill or knowledge to be learned in a way that the 
trainee understands; 

• Are interactive, i.e., actively involve the trainee throughout the training; 

• Provide opportunities for the trainee to practice the job skills or internalize the 
knowledge required by the job; 

• Provide opportunities for the trainee to perform the job skills or use the job- 
related knowledge under conditions closely resembling the job; 

• Provide opportunities for trainees to test themselves on the skills and knowledge 
taught; 

• Provide alternative learning strategies — a variety of training media — to meet the 
range of individual learning characteristics; 

• Measure performance and provide results to the trainee and to the instructor based 
upon specific job competencies; and 
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CHARACTERISTICS OF A GOOD INSTRUCTIONAL SYSTEM (continued) 

• Use competency achievement data to improve the effectiveness and efficiency of 
the system. For example, when trainees fail to achieve a competency within a 
reasonable time, the training system is examined to see what went wrong, and 
then it is adjusted accordingly. 

Such a system: 

• Consistently produces results for which the system was designed; 

• Does not leave you dependent upon the skills of one person; 

• Is replicable; 

• Capitalizes on the range of skills and unique talents of various people; 

• Places responsibility for learning with the learner, and responsibility for the 
learning environment and management with the instructor; 

• Uses what is known about how adults learn; 

• Provides a means for adjusting itself to better meet present needs and keep itself 
updated. 

Source: David Barbee, Ph.D., formerly with the Center for Advanced Learning Systems, 
U.S. Department of Labor. 
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SPECIAL FEATURES OF COMPUTER ASSISTED INSTRUCTION 



• The branching capability of the computer permits instruction to be individualized 
to the learners * needs. 

• Computer software can create interactive learning environments in which the 
learner is actively involved. 

• Computer-based drills and tutorials may be learner controlled and learner paced. 

• Computer-generated drills can provide immediate feedback and systematic 
reinforcement. 

• Computer-based instruction enables adult learners to be more self directed in their 
learning: doing their own diagnoses, choosing lessons, and scheduling their own 
tests. 

• Computer-based instruction allows adult learners to make mistakes privately and 
take advantage of the computer's infinite patience. 

• The computer can provide objective feedback about progress. It does not make 
judgments on the basis of age, sex, or race. 

• Computers can promote socialization. When two or more learners work together, 
they often help and support each other. 

• Computer applications in instruction and program management can free instructors 
to devote more time to coaching, counseling, and other "strictly human" 
endeavors. 

• Computers for adult learners promote educational equity by providing access to 
new technology not available during earlier school experience. 

• Computer assisted instruction works particularly well with students who need 
remedial work in basic literacy skills. 

• The combination of these features produces intrinsically motivating instruction; 
most adult learners are attracted to it. 
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COMPARING INDIVIDUAL AND COMPREHENSIVE LEARNING SOFTWARE 


PROGRAMS 






Software Features 


Individual Programs 


Comprehensive Programs 


size 


usually on one aisK 


UoUdiiy uh liiduy uioro ui un 
hard disks or on a mainframe 


student records 


no record keeping system 
or minimal record keeping 


record keeping system tracks 
each student's progress 


capacity 


accommodates one student 


accommodates many students 




at a time 


at one time 


interaction 


not linked to other 
programs 


linked to other programs 


coverage 


teaches a specific skill 
or set of skills 


teaches many skills; often a 
complete reading or math 
program 


testing 


usually no testing 


may include testing: 
diagnostic, progress, and final 


teacher role 


usually requires greater 
involvement 


usually operates in place 
of the teacher 



Step 3: Review existing training resources 



Once you've created your guidelines, you're ready to review existing training resources. 
This involves both looking at a program's features to determine whether they match your 
needs, and rating its characteristics. 

A courseware review form should be developed to record and summarize the features of the 
resources you review. CertainTeed's review form noted the type, media, difficulty level, 
and curriculum areas of the courseware, as well as the courseware's hardware requirements. 
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CERTAINTEED COURSEWARE REVIEW FORM 



Courseware Title: 

Publisher: 

Address : 

Order Phone: Technical Help Phone 

Price : 

Program Information 

Type: Drill and Practice Simulation Tutorial Game Instruction 

Media: CBI Print A/VT IT IVD CD-Rom CR SG OJT_ 

Difficulty Level: ABE (0-4) ABE (5-8) GED 
Curriculum Area: Life Skills Spelling Mathematics 

Problem Solving Instrumentation 

Reading Science Health Analytical 

Thinking Writing Communication 

Computer Literacy Learning 

Description/Comments 



Additional Requirements if CBI 

Type of Computer: Apple_ IBM 

Type of Disc Drives: 3 1/2 5 1/4 

Memory: 48 64 128 256 512 

Peripherals Needed: Printer CD-Rom IVD VCR 
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You should also develop an evaluation form to rate the program's features vis-a-vis the 
guidelines you created. CertainTeed's evaluation rated programs' content characteristics, 
instructional design characteristics, technical characteristics, and cost. 
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CERTAINTEED EVALUATION CRITERIA FOR LEARNING SYSTEMS 


Name of Software/Courseware 












i. Content Characteristics 
















5 


4 


3 


2 


1 


a. 


Content is relevant to adults 


□ 


□ 


□ 


□ 


□ 






5 


4 


3 


2 


i 


b. 


Content may be changed or added to 


□ 


□ 


□ 


□ 


□ 




by an instructor 










Score 


2. Instructional Design Characteristics 
















5 


4 


3 


2 


1 


a. 


Learning objectives support CertainTeed 


□ 


□ 


□ 


□ 


□ 




competency requirements 
















5 


4 


3 


2 


1 


b. 


Format is challenging, not frustrating 


□ 


□ 


□ 


□ 


□ 














Score 


3. Technical Characteristics 
















5 


4 


3 


2 


1 


a. 


Program may be entered and exited 


□ 


□ 


□ 


□ 


□ 




with ease 
















5 


4 


3 


2 


1 


b. 


Screen directions are concise, readable, 


□ 


□ 


□ 


□ 


□ 




and adequate 
















5 


4 


3 


2 


1 


c. 


Adult learners can operate the programs 


□ 


□ 


□ 


□ 


□ 




independently 
















J 


A 

4 


3 


z 


1 
1 


d. 


Sound and graphics enhance the program; 


□ 


□ 


□ 


□ 


□ 




are not demeaning 
















5 


A 

4 


3 


I 


1 


e. 


Third party courseware may be added 


□ 


□ 


□ 


□ 


□ 




to supplement the program 
















Yes 


No 








f. 


Program includes an authoring system 


□ 


□ 










If no, give lowest rating. If yes, rate 


5 


4 


3 


2 


1 




in terms of quality and ease of use. 


i — i 


U 


i — i 

l_J 


u 


n 
U 






5 


4 


3 


2 


1 


g- 


Quality of supplemental materials 


□ 


□ 


□ 


□ 


□ 














Score 






5 


4 


3 


2 


1 


4, Cost 


□ 


□ 


□ 


□ 


□ 














Score 


Rating Scale: S-Excellent; 4 = Very Good; 3 = Acceptable; 2 


= Marginal; 1 =Unacceptable 




Total 
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After evaluating the materials, you should develop a system for ranking the materials 
according to the degree that they meet your needs. To do this, you might want to weight the 
various characteristics you rated on the evaluation form before adding them up. Rank 
ordering the various products will indicate which ones you should obtain more information 
about. 

Step 4: Select existing training resources 

The rank-ordered list of training systems can be narrowed down by focusing on the key 
characteristics of learning systems appropriate for your manufacturing setting. 
In order to make the final selection, you might want to choose the three to five most 
appropriate learning systems for your plant and arrange for a final evaluation by the training 
program's steering committee. Invite the retail companies of the top ranked systems to make 
presentations and provide hands-on demonstrations for the committee. Then, solicit 
committee members 9 opinions and make recommendations on the learning system your plant 
should buy. 

CertainTeed found that a great deal of their basic skill training needs could be met using an 
off-the-shelf learning system, though employees much preferred the basic skills learning 
materials that were later developed from job-linked resources. Most training for functional 
skills (applications of basic skills on the job) required more traditional methods such as small 
group and classroom instruction, some of which could be provided by CertainTeed's 
Corporate Training Department. Most technical training needs had to be met through 
customized curriculum and structured on-the-job training. 
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TASK 2: DEVELOP CUSTOMIZED TRAINING 



Customized training enables ycu to pinpoint training on your skills needs. Every 
manufacturing plant is unique, and customized training ensures that your unique needs are 
met. Well-designed customized instruction developed in the context of the job should be the 
core of your precision strike training program. Developing customized training involves: 

• Selecting a vendor if you require outside assistance; 

• Developing the framework for customized training within the company; 

• . Determining the lesson content; 

• Dividing the lesson content into units; 

• Adjusting the curriculum for ability levels; 

• Choosing the instructional media; 

• Developing training lessons; and 

• Developing and testing assessment tools. 

Step 1: Select a vendor for the customized training that will be developed outside of 
the company 

You may choose to hire an outside vendor to develop your customized training programs. 
There are a number of important features that you should look for when you are selecting a 
firm for this task. These include: 

• The firm's software should have an authoring system that is easy to use. An 
authoring system allows you to customize the lessons and place them in the context of 
participants' jobs. 

• The firm should have a history of developing training materials in the "context" of the 
workplace. This is key because precision strike training is a functional context 
training program. 

• The firm's business goals and strategy should make it possible for the company to 
develop programs for you at a much lower cost than other providers. 
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CertainTeed decided to use the firm it selected to create three training programs: 

• A one-hour orientation program which would give all employees an overview of 
company policy and procedures and the manufacturing process. 

• A two-hour course on statistical process control as related to the CertainTeed roofing 
products group manufacturing process. 

• A course on instrumentation designed to review and simulate the CertainTeed 
manufacturing process by showing proper use and settings of all gauges, switches, 
and panels. 

Step 2: Develop the framework for the customized training that will be developed 
within the company 

You might also choose to develop customized training in house. Both classroom and 
structured on-the-job training should be an integral part of any customized job training 
program. Both types of training should target on the specific skill needs of specific 
individuals - the essence of precision strike training. Structured on-the-job training is also 
valuable in providing actual practical experience. 

CertainTeed developed a customized training framework that addressed components of on- 
the-job training, priorities for training development, training material, and placement of 
individuals in training programs. 
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CERT AENTEED' S ON-THE-JOB TRAINING APPROACH 

• Structured on-the-job training was combined with formal training. Structured on- 
the-job training consisted of a designated trainer for each position, working with 
an individual trainee, and using a checklist of competencies to determine whether 
skills were learned and demonstrated. 

• For the formal training, CertainTeed, in partnership with Vance-Granville 
Community College, developed "how-to" training materials for each job. 

• The training started with those jobs that have the biggest impact on the 
manufacturing process, and moved from higher level to lower level jobs. 
Interactive video disks were used for the training, which was developed in the 
functional context of the jobs. 

• The on-the-job training was structured around a "family of jobs" concept. The 
goal was that each person holding a job would eventually train for the job(s) that 
act as supplier to him/her and the job(s) for which he/she acts as the supplier of a 
product or service. 

. For example, the coater operator family of jobs might be the dry end operator 
(supplier) and slate and surface operators (customers). 

This approach to structured on-the-job training was designed to ensure a succession 
system for key manufacturing jobs, and to ensure that training has direct and measurable 
positive impacts on the manufacturing process. 



Step 3: Determine the lesson content of the curriculum 

To determine the curriculum's lesson content, you should review the job and literacy task 
analyses and accompanying materials that are used on the job. Study the various subtasks 
and skills contained in these materials, and review the job materials to determine what skills 
must be taught. Integrate the results of these analyses, and from this material, select the key 
elements of instruction. Finally, organize the elements to form a logical presentation 
sequence. 

Step 4: Divide this content into units 

After you have determined the overall content of the curriculum, you should divide the 
curriculum into units. Units should be defined by common themes or key elements in 
thinking processes. 
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You need to consider the sequence of planned instructional activities in relation to each 
other. You should design the units of instruction to allow for continuity and sufficient 
practice time for learner mastery. Plan for continuity between topics and sessions by 
preparing an instructional unit planning chart. Do not break topics off mid-session. Begin 
by estimating the time you think is required to conduct an adequate number of instructional 
activities for each topic. If topics are too long, it may be sufficient to highlight only key 
concepts within a complex task or to spread the subtasks over several sessions. If you spread 
a topic over several sessions, identify the natural breaking points between subtopics. 



THINGS TO KEEP IN MIND WHEN CREATING INSTRUCTIONAL UNITS 
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- Insufficient learner practice, and 




- Frustration. 


• 


Too little content results in: 




- Wasted time, 




- Learner boredom, and 




- Employer dissatisfaction. 


• 


Sessions should run from 1 1/2 to 2 1/2 hours. 


• 


Consideration should be paid to conditions related to the time of day instruction is 




to be delivered; e.g., whether the session is held before or after shift. 


• 


The length of time for effective instruction varies with the complexity of the task. 



Step 5: Adjust the curriculum for ability levels 



It is extremely important that the curriculum be at the right level for the individual workers. 
If examples go over workers' heads, essential lessons may not be learned. The same end 
may result from instruction that is too simplistic for its audience: the audience may be lost 
to boredom. 

You should keep in mind that individuals learn at different rates, depending on their level of 
literacy. Workers who learn at the same rate should be grouped together. 



For trainees with lower literacy levels, you should: 

• Offer a sequence of courses leading up to skill application mastery over a longer 
period. 

• Target smaller segments of content within the time frame of a session or course. 

• Use special instructional materials and delivery techniques, such as 

- assigning more step-by-step exercises so that slower learners can understand, read, 
write, and use technical vocabulary, 

- using shorter and more focused lessons that break skills down into small 
increments, 

- using lessons that build on what trainees already know, and 

- including more practical, hands-on exercises. 

One way to adjust the curriculum for workers with low levels of literacy is to redesign the 
company forms used as learning material in such ways as: 

• Adding definitions for technical terms such as "average" or "lower control limit"; and 

• Teaching the whole number operations needed for computing averages in sequential 
mini-lessons that focus on the relationship of these operations to the statistical control 
process. 

Once they have achieved mastery, learners can tackle the actual workplace statistical process 
control forms that are used on the job. These modifications and special mini-lessons would 
not be necessary for workers with higher levels of literacy. 

For an English as a Second Language target group, the lessons should be short, focusing on 
specific skill applications for equipment operation, safety, and workplace interpersonal 
communications. Emphasis should be on learner mastery of speaking, reading, and writing 
English for priority job skill applications. For example, workers could be taught how to 
communicate with each other during breaks; and managers could study the art of asking 
questions that elicit more than "yes" or "no" answers. 

[NOTE: Some of the information for this step was taken from Literacy at Work: The 
Workbook for Program Developers, by Jorie Philippi and the National Alliance of Business, 
New York, Simon & Schuster Education Group, 1991.] 




Step 6: Choose the instructional media and format 

Instructional media include computer-assisted instruction, interactive videodisc, self-paced 
functional context workbooks, tutoring, and classroom and structured on-the-job training. 

Computer-assisted instruction works particularly well with students who need remedial work 
in basic literacy skills. The advantages of computer-assisted instruction are that it can be 
individualized to a learner's needs; provide immediate and systematic reinforcement; enable 
adult learners to be more self-directed in their learning; permit adults to make mistakes 
privately; and free instructors to devote more time to coaching, counseling, and other 
"strictly human" endeavors. 

In general, interactive videodiscs have benefits similar to those of computer-assisted 
instruction, but may have the added advantage of promoting a higher degree of active 
individual participation. Self-paced functional context workbooks allow individuals to work 
at their own speed, and have the advantage of being portable. Tutoring makes is easier to 
taiior expert advice to an individual's needs. Classroom instruction is a cost effective 
approach to providing the same information to a number of workers at the same time. 
Structured on-the-job learning is excellent for teaching sk: n s in the functional context. 

The guiding principle in choosing instructional media is to select a form that most nearly 
resembles the context in which the skill applications are used on the job. 

For the Precision Strike training at CertainTeed, a computer-assisted instructional curriculum 
was custom developed for each job in the program. Some components of the individual 
lessons were derived from the bank of lessons available in the integrated learning system 
purchased by the company. A workbook, designed to teach the functional applications of 
basic skills, was also developed for each job in the program, using the actual reading, math, 
and writing materials used on the job. 

CertainTeed also developed a series of five three-hour classroom sessions. The topics for 
these sessions were: 

• Overview of CertainTeed' s Commitment to Quality and How it Impacts Each 
Production Crew Member 

• Using the Process Monitoring System 

• Statistical Process Control Math 

• Using Charts and Graphs 

• Problem Solving Skills 
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Step 7: Develop training lessons 

There are a number of things that you should think about in developing training lessons. The 
lessons should: 

• Discuss subjects that are related to the lesson and common to the experience of the 
learners. This tactic 

- builds motivation, 

- creates mental hooks on which learners can hang new information, and 

- validates the adult learner's wealth of personal knowledge and life experiences. 

• Build knowledge of new or familiar technical terms. One way to do this is to 

- list words on flip charts, 

- ask learners to define the terms, and 

- develop one- or two-word definitions that are easy to remember. 

• Take the learners slowly through the job tasks and subtasks by 

- stating aloud the steps a competent worker uses to successfully perform the job 
tasks, 

- presenting concepts in manageable chunks, and 

- carefully explaining each small step until learners are proficient at all of them. 

• Provide customized skill application exercises for guided practice. These exercises 
should 

- allow learners to try out skills modeled during the lesson, 

- require the learner to apply the skills to a job scenario, 

- allow ample opportunity for learners to ask questions, 

- provide extended practice and supplemental instruction in closing individual skill 
gaps, and 

- provide lots of positive feedback. 
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• Provide exercises for applied practice. You should 

- identify commercial resource material that supports the lessons being taught, and 

- integrate these resources into the program as needed. 

• Close with a summary of the concepts taught. This summary closing 

- consists of a five-minute summary activity for each lesson, 

- facilitates the transfer of learning, 

- asks learners to use their own words to put what was learned into summary 
statements, which reinforces their conceptual understanding and builds a sense of 
ownership of the skills, and 

- solidifies learners' understanding and increases skill retention. 
Step 8: Develop and test assessment tools 

Testing plays a crucial role in a competency-based approach to learning. Four of the most 
important uses of tests include: 

• Diagnosing the competence of new learners or program participants; 

• Providing immediate feedback to learners during the learning process; 

• Assessing learners' mastery of each task; and 

• Evaluating the effectiveness of training materials. 

The type of tests most appropriate to a competency-based approach to learning are called 
criterion-referenced tests. The name is derived from the purpose of the test: to find out 
whether the criteria stated have been achieved. 

Criterion-referenced tests are constructed using the actual tasks that a learner must perform 
as the source of the tests. They are always performance based. That is, criterion-referenced 
tests compare the actual performance of the learner against the criteria stated in the task. 

As part of the Precision Strike training project, CertainTeed developed criterion-referenced 
assessment instruments for all production jobs. CertainTeed's goal was to identify the 
reading, math, and writing requirements for each job through the literacy task analyses, and 
then develop an assessment tool that would accurately measure workers' competence in 
performing these literacy requirements. The questions on each test were derived from the 
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job and literacy task analyses conducted for the training project. CertainTeed established 100 
percent as a minimum acceptable score for mastery. 

For example, a coater operator must be able to accurately calculate the mean of a set of four 
bundle weights with no addition, subtraction, or division errors. The CertainTeed 
assessment instrument contains test items that provide the coater operator with a set of four 
bundle weights and requires that the operator calculate the mean of those numbers with no 
errors. 
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SAMPLE CRITERION-REFERENCED QUESTIONS 

The following questions were taken from the coater operator's pre- and post-tests for the 
Precision Strike project. For each question the functional skill and literacy application is 
noted. 

Question: The weight of an empty test tube is 31.65 grams. The weight of a test tube 
filled with coating is 45.73 grams. What is the weight of the coating? 

Functional Skill: Calculating the weight of the coating. 

Literacy Application: Subtracting decimals. 

Question: According to the Process Targets and Specifications chart, the target 
temperature for Hearthstead raw coating is 490 degrees. Look at the following 
temperatures and circle the ones that are less than the target. 

275° 300° 500° 175° 600° 235° 

Functional Skill: Obtaining the correct temperature for Hearthstead raw coating. 

Literacy Application: Determining greater than and less than. 

Question: If the target temperature for raw coating is 495 degrees plus or minus 10, 
what is the maximum allowable temperature? 

Functional Skill: Calculating the maximum allowable temperature for coating. 
Literacy Application: Adding and subtracting whole numbers. 

Question: The coater operator must know how to calculate the average. Calculate the 
average of the following sets of numbers. 

85 87 89 86 

210 220 190 205 

Functional Skill: Calculating the average. 

Literacy Application: Dividing whole num bers. . 
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SAMPLE CRITERION-REFERENCED QUESTIONS (continued) 

Question: If a line is producing 7.85 squares/minute, how many squares are being 
produced in .75 hours? 

Functional Skill: Calculating the number of squares produced during a specified time 
period. 

Literacy Application: Multiplying decimals. 



Question: Refer to your CertainTeed Coater Operator Training Manual, line 2. Find the 
section with the heading Coater Tasks. Carefully read the page and complete the 
following statement. 

The main task of the coater operator is: 

to inspect the quality of the product to ensure that it is within specifications. 

to make sure all the tools and equipment needed to perform the coater tasks are 

available. 

to apply an asphalt and limestone coating to the top and bottom of the continuous 

mat. 

Functional Skill: Understanding the tasks of a coater operator. 
Literacy Application: Critical reading skills. 



Question: Read the attached Safety Memo and write a short summary of it. 
Functional Skill: Understanding the safety regulations. 
Literacy Application: Summarizing a memo and writing skills. 



CertainTeed used the assessment tools developed for each job in the following ways: 

• As a diagnostic tool or, in their terminology, a pre-test. The pre-test information was 
used to determine the current level of competence for each worker. It served as the 
basis for assigning computer and workbook lessons for each individual worker. If a 
worker did not show mastery in a particular area, lessons were assigned to help that 
worker develop the skills needed to achieve mastery. 
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• As a means of providing feedback to program participants on their level of 
competency. The pre-test results were shared with each program participant, and a 
training plan was provided that laid out exactly what the participant needed to 
accomplish and master over the life of the program. 

• As a tool in assessing mastery of each task. The same instrument used for the pre- 
test was also used as a post-test to measure the competency level of each program 
participant after participation in the Precision Strike training project. The differences 
between pre- and post-test mastery scores were one measure used to assess individual 
learning gains. 

In undertaking the test development process you should: 

• Determine exactly what should be tested. Always remember that the test should 
match the task. Ask yourself: Exactly what should the learner be required to do to 
demonstrate competence in the task? The answer to this question describes the testing 
situation called for. 

• Conduct the job and literacy task analyses for each job or task for which you plan to 
develop training. Make sure that the results of your analyses are written up using a 
performance or competency-based approach. For example, an inspector must be able 
to: 

- calculate the range of a set of four bundle weights with no errors, and 

- measure the length and width of a shingle to within ± 1/16 of an inch. 

• Determine whether process, product, or both is critical. The process is how the 
learner performs the task; the product is the end result. Sometimes, how the task is 
performed is critical to attaining competence; other times, the finished product is what 
is important, and in some cases, both process and product are critical. You should 
assess what is required for job competence. 

• Construct the items to be included on the test. The heart of a good performance test 
is the specific items for which a rating will be given. Where process is important, the 
test items will be based on procedural steps identified during the job and literacy task 
analyses. If the product or outcome is important, your test item should reflect the 
desired outcome. Ask yourself this question: What essential characteristics must the 
outcome have for the learner to be judged occupational competent in producing the 
product? 

• Develop a test question or activity that will accurately measure an individual's ability 
to actually perform a particular task. For example, CertainTeed constructed the 
following test item to measure an inspector's ability to calculate the range of a set of 
four bundle weights: 



- Given the following four bundle weights, calculate their range. 

83.7 lbs 82.6 lbs 81.8 lbs 81.2 lbs 

Place questions in the appropriate sequence. The CertainTeed assessment instruments 
were designed using a building block sequencing approach. For instance, in order to 
be able to calculate the range, learners must be able to subtract. The tests assessed 
their ability to subtract prior to being given the question on calculating the range. If 
learners were not able to calculate the range, yet had shown mastery on the 
subtraction question, the assumption was that they did not know the formula for 
calculating the range. 

Determine how items will be rated. The specific items included in the assessment 
tool will have to be rated by the instructor or whoever is evaluating the learner. Two 
common methods are a rating scale and a checklist. 

For example: To assess the quality of a wrapped bundle of shingles, CertainTeed 
would develop a set of questions that measured characteristics such as: 

1 . Are the ends of the wrap aligned properly? 

2. Are the packages sealed? 

3. Is the end code readable? 

Question 1 could be judged using a rating scale of 1 to 5 ranging from acceptable to 
unacceptable. 

Questions 2 and 3 could be judged using a checklist with a yes or no answer. 

Most of the test items constructed for CertainTeed's assessment instruments were 
concrete and discrete. The learner either answered correctly or incorrectly. A 
correct answer showed mastery and an incorrect answer showed non-mastery. All test 
items were grouped in broad literacy skill application categories. For example, the 
literacy task analysis indicated that all operators must be able to add whole numbers. 
The test items corresponded to how learners add whole numbers on their job. If the 
learners answered correctly, they were deemed competent in that area. If items were 
missed, they were deemed not competent. 

Determine the minimum acceptable score for mastery. Companies vary in the 
minimum score they require learners to obtain in order for them to be considered 
competent. Some accept 80 percent or lower, while others require 100 percent. 
CertainTeed used 100 percent as its score of competency, though this score is not 
universally adopted. In a lean manufacturing environment where quality of product is 
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paramount, there is no room for less than 100 percent accuracy in all of the reading, 
math, and writing skill applications required on the job. 

Write directions for the learner. The directions to the learner should be worded 
carefully so that they explain: 

- the purpose of the test, 

- the general testing situation, 

- exactly what the learner is to do, 

- any special restrictions or cautions, 

- any time limit, and 

- how mastery will be determined. 

Assemble the test and try it out. It is a good idea to try out the test with a fellow 
instructor and one or two learners before using it to evaluate learners. Have these 
individuals look at the directions and each test item for clarity, and check for vague 
and poorly worded items. In a manufacturing setting, it is extremely important to 
make sure that each item reflects the actual job situation and that the commonly 
accepted vernacular is used. Make any revisions that are needed. 

At CertainTeed, the tests were constructed by the project consultant and the Vance- 
Granville Community College instructor, neither of whom are expert in all of the 
production job tasks. They found that in some instances they were testing the right 
skill application, but had worded the items in a way that was confusing to the worker. 
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DEVELOPING CRITERION-REFERENCED PERFORMANCE TESTS 

Here are some tips to use in developing assessment tools in a lean manufacturing 
environment: 

• The test should assess the tasks workers must perform. 

• The actual behavior called for in the task should also be required for the 
performance test. 

• If a process is critical to performing the task competently on the job, items must 
be included that assess how the learner performs the task. 

• If producing a completed product is important to being competent, the outcome 
must be evaluated critically. 

• The test should have the learner repeat the task several times, each time under 
different conditions, and during different time periods. 

• The test should measure how each individual learner performs the task 
independently, unless the task is always performed by a group on the job. 
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Ill, IMPLEMENTATION 



TASK 1: CONDUCT PRELIMINARY ACTIVITIES 

A great deal of preparatory work has to be done before you can actually put employees 
through your training program. These activities include: 

• Establishing the program and its physical facilities; 

• Hiring and preparing staff; and 

• Recruiting program participants. 
Step 1: Select a name for the program 

An appropriate name can be critical in establishing your program's identity. The name of 
CertainTeed's first literacy program was Skills and Knowledge in Lifelong Learning, or 
S.K.I.L.L. The name was significant because it projected and reconfirmed CertainTeed's 
commitment to and responsibility for the "lifelong learning" of its employees. 

In selecting your program name, you should think carefully about the image you are trying to 
project to your audience, both employees and senior management. A note of caution: many 
employees are not receptive to names that denote remediation, basic skills, and/or literacy 
because they feel such names stigmatize them as having failed at earlier lessons. 

Step 2: Establish physical facilities 

Obviously, your training program cannot take place without some physical facility. There 
are many factors that should go into choosing a site, including availability and suitability for 
instructional purposes, and convenience to employees. 

CertainTeed chose to construct a new learning facility on its factory grounds. This enabled it 
to provide a facility that was expressly designed for training and available to employees at all 
times. The training center building is 68 by 24 feet and provides 1,632 square feet of space. 
The heart of the center is ten computer work stations, including three stations set up to use 
interactive video media and CD Roms. In addition to the computer lab, there is an office for 
the training administrator and a conference room/instructor office. The interior of the center 
is comfortably furnished and decorated and the exterior is attractively landscaped. Because 
CertainTeed manufactures building materials, it was able to use its own siding, windows, 
doors, insulation, and roofing shingles in constructing the building. 

The training center is open on a drop-in basis 24 hours a day. Workers are encouraged to 
spend a minimum of three hours per week studying at the center. The proximity of the 
center to the workplace makes training participation both convenient and accessible. 
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Your training facility does not have to be state of the art in order to serve its purpose. If 
you cannot make such a large investment, you could seek out a room at your facility that can 
be made available at regularly scheduled times. If this is not possible, you could look into 
borrowing space from local schools or churches. Remember to take all three factors -- 
availability, suitability, and convenience ~ into consideration when selecting a site. The lack 
of any one of the three could sink your program. 

Step 3: Develop a staffing plan and secure staff 

Your staffing plan should be shaped by the following considerations: 

• The type of material to be conveyed (general vs. technical); 

• The format being used to deliver the training (e.g., self-paced exercises, group or 
classroom instruction, on-the-job training); 

• The availability of in-house staff; and 

• The total budget available to hire trainers. 

The instructors for your precision strike training program can be drawn from a variety of 
sources, including consultants, community college and adult education teachers, your own 
human resources department (if you have one), and managers and line employees expert in 
particular skills and tasks. Which of these individuals is most appropriate will depend on the 
considerations outlined above. 

Consultants and community college and adult education teachers (and possibly individuals 
from your own human resources department) are likely to be skilled in training techniques 
while not knowledgeable about your manufacturing operations and particular skill needs. 
Managers and line employees, on the other hand, are likely to have the appropriate 
specialized knowledge, but be unskilled in sharing that knowledge with others. 

It is important not to underestimate the problems involved with selecting trainers out of this 
last group. While expert managers and line employees bring a wealth of talent to the table, 
they also bring their own set of problems. Besides not being trained as trainers, their 
schedules are usually packed, and they have little time to prepare for training responsibilities. 
These time constraints sometimes force them to put off working on the training until the last 
minute. This can result in sessions that are disorganized and delivered as monologues — not 
the most effective learning environments. 

The best hiring plan is one that plays off the strengths and compensates for the weaknesses of 
each of the groups of potential instructors. Whenever possible, skilled trainers should be 
paired with substance experts to develop and deliver the training. This pairing ensures that 
both the what and how of lesson delivery is covered adequately. 
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In ideal situations, both the skilled trainer and the content expert would be involved in all 
group or classroom instructional exercises, with the trainer leading the sessions and the 
content expert serving as back-up to answer difficult or detailed questions. However, if only 
one instructor can be provided in a classroom setting, that individual should be a skilled 
trainer. Good technique is critical for maintaining participants' interest and concentration. 

Content experts can best serve on their own in situations where instruction is one-on-one. 
This includes serving as a resource person to answer questions that arise as individual 
program participants work through self-paced workbook or computer exercises. It also 
includes serving as a trainer or mentor for on-the-job training exercises. 

CertainTeed hired a full-time training supervisor responsible for all plant-level training 
activities including the administration and management of the Precision Strike program. The 
training supervisor also has lent his expertise as a former CertainTeed production worker and 
supervisor to the development of job-related basic, functional, and technical curriculum. 

Two instructors, one full time and one part time, staff the training center. They provide 
instruction, develop curricula, handle administrative work, and provide user assistance. 
Although both are employees of Vance-Granville Community College, CertainTeed shares 
salary expenses by paying 40 percent of the full-time position and 100 percent of the part- 
time position. 

Both instructors have college degrees and previous experience in adult basic education and 
General Educational Development (GED) instruction. Prior to the CertainTeed program, 
neither had specialized experience in workplace literacy. 

An outside educational consultant with extensive workplace literacy experience was 
responsible for overall coordination, implementation, and technical oversight and guidance 
for operations and curricula. The consultant conducted the up-front research, structured the 
program, developed curricula, and designed and conducted the final evaluation. 

In-house support from various departments within the Oxford plant has been generously 
available. The plant computer expert has been an invaluable resource and provides technical 
assistance with computer hardware and software on an as-needed basis. The members of the 
Plant Manager's staff, particularly the Plant Superintendent and the Quality Assurance 
Manager, have been extremely helpful and supportive in all administrative and operational 
aspects of the training program. 

A workplace literacy coordinator for Vance-Granville Community College provides outside 
support in the areas of coordination with Vance-Granville Community College, instructor 
supervision, and development of curricula and instructional materials. Her role has been 
significant in the successful coordination and use of resources between program partners. 
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In addition, the Vance-Granville Community College Business Services Coordinator has 
played an extremely important role in the development and implementation of specialized 
classroom instruction such as the Working Program, Frontline Leadership, Total Quality 
Transformation, and maintenance worker training. 

Step 4: Train Staff 

No matter who you select as your trainers, they will need some preparatory training 
themselves. Outside trainers will need to learn the technical operations of your company, 
and inside experts will need to learn effective instructional techniques. 

CertainTeed's program staff generally developed their workplace literacy expertise on a daily 
learn-as-you-go basis, with guidance and support from the workplace literacy consultant. 
The workplace literacy consultant also played an important role helping to develop the 
capacity of the Vance-Granville Community College instructors. Several activities were 
conducted in concert with Vance-Granville Community College to enhance the college's 
ability to develop, design, deliver, and evaluate workplace literacy programs. 

The first of these efforts was a day-long seminar with the theme: Strengthening the 
Workplace through Basic Skills Education. Presentations and discussions at this seminar 
focused on "Workplace Literacy: A to Z" and "Creating the Meaningful Educational 
Experience." 

The consultant also conducted a series of seven workshops for community college instructors 
to train them in the specific skills needed to help local industries develop functional context 
workplace literacy training. These workshops were funded by the National Institute for 
Literacy as part of a grant to Vance-Granville Community College, CertainTeed, and the 
National Alliance of Business. 
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TASK 2: OPERATE YOUR PRECISION STRIKE TRAINING PROGRAM 

How efficiently you implement and how smoothly you operate your precision strike training 
program will have a great effect on how successful it is. Program operation can be divided 
into five steps: 

• Conducting a program orientation; 

• Initiating a recruitment and enrollment campaign; 

• Assessing participants' skills; 

• Developing individualized training plans; and finally 

• Training participants. 

Step 1: Conduct a program orientation 

The first part of any training program should been an orientation. This orientation should be 
designed to inform employees about the program what it is intended to do, how it will 
operate, and how it will benefit employees. The orientation should also try to make 
employees comfortable about participating in the program. 

Prior to the opening of the training center, small groups of CertainTeed employees were 
brought over to the center for a two-hour program orientation. All CertainTeed employees 
participated in the orientations. Program rules, policies, and procedures were explained in 
detail. In addition, employees were given a chance to actually sit down and try out the 
computers. 

CertainTeed' s orientation process continues for new employees as well. All new employees 
are provided with an intensive orientation program focusing on ensuring that they understand: 

• The roofing products group (RPG) goals of quality and safety; 

• The overall RPG production process; 

• CertainTeed, RPG, and plant policies; and 

• What is expected of them and what opportunities are available to them. 
Step 2: Initiate a recruitment and enrollment campaign 

If you are going to offer your employees the option of participating or not participating in 
your precision strike training program, you will have to recruit trainees. Program 
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participants can be recruited through a wide variety of means, including personal contact, 
supervisor encouragement, notices, flyers, start-up videos, production crew meetings, 
departmental meetings, personnel department referrals, posters, newsletters, or self-referrals. 

CertainTeed made participation voluntary for incumbent workers, but mandatory for new 
hires. However, promotion at CertainTeed is tied directly to participation. Job bids are only 
given to program participants, where participation is defined as 12 hours per month. 

CertainTeed offered additional incentives beyond the program's link to promotions. For 
every two hours of training, employees are compensated for one hour of straight time. Other 
incentives and awards include: 

• Monthly drawings for a $15.00 dinner certificate for those employees with 16 or more 
hours of participation in a month; 

• A certificate of recognition for participants completing certain curricula; 

• A certificate of recognition and a $50.00 dinner certificate or a jacket for participants 
completing required training levels; 

• A plaque plus a $50.00 dinner certificate for participants attaining a General 
Educational Development (GED) certificate; 

• From time to time, gifts such as caps, pens, and mugs with the program's logo for 
employees with 12 hours of monthly participation; 

• Recognition in the program newsletter and other organizational publications for 
participation; and 

• Recognition at the annual company awards banquet for program participants with 
outstanding achievements. 

Step 3: Assess participants' skills 

While you should have already assessed employees' skills as part of your initial research and 
analysis (Section I, Task 3), you may need to conduct additional assessments now. The 
initial assessments could have been given anonymously and/or to a sample of employees to 
determine the general level of employee skills and thus help shape curriculum development. 

Assessments at this stage should be given to the particular individuals enrolling in your 
precision strike training program and should be used to customize instruction to meet each 
individual participant's needs. Obviously, if you have already assessed your participants and 
kno'A, how each participant scored, you do not need to re-assess them now. You can just use 
the original assessments for a new purpose. 
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Individual assessment is a key activity in CertainTeed' s Precision Strike training program. 
Each individual who participates is given the Test of Adult Basic Education. In addition, as 
part of the grant from the National Institute for Literacy, CertainTeed developed criterion- 
referenced assessment tools which allow for a more accurate and discrete analysis of the 
literacy skills required for a particular job. When designing your precision strike training, 
you should also explore the recently-developed Tests of Applied Literacy Skills to see how 
well they match up with your company's needs. 

Step 4: Develop individualized training plans for program participants 

The results of participants' assessments should be used as a guide in developing 
individualized training plans for each participant. These plans should be designed to close 
the gap between participants' literacy ability levels and the skill requirements of the 
workplace. 

At CertainTeed, the training plans for program participants also include a standard set of 
instructional units addressing the functional and technical skill requirements of each 
participant's training level. 

Precision Strike training participants are counseled by the Vance-Granville Community 
College instructors assigned to the training center about their test results and the curriculum 
requirements for their training levels. During the initial counseling session, the instructor 
tries to identify any barriers not apparent from the test scores that will interfere with each 
individual's ability to participate in the program. 

Step 5: Train participants 

Once individualized training plans have been developed, you are finally ready to begin 
training participants. 

Initially, CertainTeed simplified its training program by identifying and creating five distinct 
training levels based on the complexity of the job tasks and the literacy skill applications 
associated with each of those tasks. Setting these training levels allowed CertainTeed to 
establish common curriculum requirements for similar jobs. In addition, training levels 
provided a structured mechanism for measuring employees' progress. Each training level has 
a prescribed curriculum covering skill development in five broad areas - basic, functional, 
technical, work maturity, and physical. 
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CERTAINTEED JOBS GROUPED BY TRAINING LEVELS 
Training Level I Training Level IV 


All New Hires 
Laborers 

Third and Fourth Hands 
Clean-up 
Utility Operators 
Grand Manor Operators 


Saturator/Coater Operators 
Granule/Top Slate Operators 
Boiler/Stillyard Operators 
Inspectors 
Lead Relief 
General Operators 


Training Level II 


Training Level V 


Hallmark Operators 
Loaders (Warehouse) 
Supply/Checkers 
Supply/Service (Membrane) 
Unloaders 


Maintenance Electricians 
Maintenance Machinists 
Maintenance Mechanics 


Training Level III 




Wrapper/Pal let izer Operators 
Dry End (Felt/Membrane) Operators 
Surface Operators 
Cutter/Winder Operators 
Service Supply (Yard) 





When employees enter the Precision Strike training program they are automatically enrolled 
in the curriculum associated with the training levels for the jobs that they occupy. Prior to 
applying for jobs in higher training levels, employees must complete all of the training 
requirements for their current training levels. This is consistent with the overall purpose of 
CertainTeed's workplace literacy program: that all workers possess the skills they need to 
perform competently in the jobs they presently occupy, and are prepared to deal effectively 
with the new technologies, tools, and manufacturing processes. 

Over time, the rule will change, and employees will not only have to complete the 
requirements for their current training levels, but also the requirements for the training levels 
of the jobs for which they are applying. 

CertainTeed's training program includes a variety of instructional media, including computer- 
assisted instruction, structured job skills training, cross training, classroom instruction, and 
individual tutoring. 
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Computer-assisted instruction is a major component of the Precision Strike training 
program. Self-paced instruction is provided through the ten computer work stations, video 
and audio playback units, programmed and self-instructional written material, and reference 
and resource material provided in the center. The advantages of this type of instruction for 
adults are well documented in the workplace literacy literature. The major disadvantage of 
this type of instruction is the inability to custom design curriculum in a functional context. 

Structured job skills training is a key part of the Precision Strike training program delivery 
system for the development of technical job skills. Traditionally, CertainTeed has used an 
unstructured approach in training new workers. The major problem associated with the 
unstructured approach was inadequate and inconsistent training. In many cases the job was 
learned incorrectly, and training time was insufficient. 

The approach implemented through the Precision Strike program has the following key 
features: 

• The selection and training of competent workers as designated trainers (all trainers 
participated in a train-the-trainer program designed to help them develop the skills 
needed to be a competent trainer); 

• The development of performance-based checklists which provide trainers with a tool 
to judge the performance and competence of trainees by job task; 

• The development of a training approach based on a simple, three-fold process of 
actually observing and performing the job tasks to demonstrate competent 
performance, the three steps being: 

- the trainer demonstrates the task, 

- the trainee explains and tells the trainer about the task, and 

- the trainee performs the task; and 

• The development of tools and approaches that support and reinforce the performance 
of the job tasks, including technical training manuals and classroom training. 

Structured job skills training (SJST) is being provided first to workers in their present jobs. 
In addition, it is being used to train individuals who bid on and are chosen for new jobs. 
These new workers are not allowed to perform their new jobs independently until the SJST 
trainers decide that the individuals are competent in all critical job tasks. 

Cross training will also become a critical component of the Precision Strike training 
program. The intent of cross training is to ensure that individuals on a crew are trained to 
perform more than one job. To create the cross training component, CertainTeed grouped 
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key manufacturing jobs into job families based on their relationships to each other in terms of 
"supplier" and "customer." Ultimately, CertainTeed plans to train all workers to 
competently perform the jobs in their "families." The training approach used will be based 
on the structured job skills training approach described above. 

Classroom instruction has been used to supplement the core program at CertainTeed. This 
instruction has ranged from small, short-term, targeted sessions on basic skill areas such as 
fractions and decimals to extended functional and technical skills training programs 
conducted by Vance-Granville Community College. Specifically, the following classroom 
instruction has been offered: 

• OJT Train-the-Trainer 

• Fractions 

• Presentation Skills 

• Frontline Leadership for Supervisors 

• Working 

• Problem Solving 

• Key Product Quality and Productivity Issues 

• The Process Monitoring System 

• SPC Math 

• Pareto Charts 

• Total Quality Transformation 

Individual Tutoring is available upon request on an as-needed basis for any program 
participant who needs individual attention or remedial assistance. Training center staff 
provide this extra help. 

Part of the process of training participants is monitoring their progress and providing them 
with feedback and appropriate assistance. You can check participants' progress in a number 
of ways. Many off-the-shelf computerized instruction programs have management modules 
that allow you to track participants' efforts. Workbooks can be periodically collected and 
reviewed. In addition, assessment tests can be administered from time to time. The purpose 
of any of these monitoring methods is not to grade employees' work, but to let them know if 
they ar? having problems and to help them overcome the problems as soon as possible. 



74 

85 



TASK 3: SUPPORT THE PROGRAM'S OPERATION 



Once your program is up and running, you can only take a brief breather before you need to 
occupy yourself with its "care and feeding." You need to monitor your program at all times 
and make any adjustments necessary to ensure its success. 

You also need to continue to promote your program to ensure that: 

• Employees know about and participate in it; and 

• Senior managers know its value and continue to support it. 
Step 1: Monitor and adjust program operations 

Once you have begun to train participants, you need to 1) monitor and solicit feedback on 
program operations and training content and delivery, and 2) assure program quality. 

Program monitoring and quality assurance can be undertaken through informal, on-site, on- 
going processes. Multiple sources of information, data, and feedback can be used to compile 
both qualitative and quantitative data on all aspects of implementation and operation of your 
training program. Potential data sources include interviews, observations, anecdotal 
evidence, focus group discussions, instructor input, questionnaires, surveys, written reports, 
administrative records, and computer-assisted instruction management systems. 

As a result of on-going monitoring and feedback, CertainTeed's training program was 
adjusted and improved on an as-needed basis. Specific program components v/ere looked at 
with an eye toward making each as workable, practical, and efficient as possible. 

The purpose of this monitoring and feedback process was to 1) assess process and outcomes, 
and 2) identify red flags, stumblug blocks, barriers, and glitches in program operations, 
processes, and equipment. 

Some areas that received attention were: practice and follow-up for the use of new skills, 
practice time, supervisor involvement and support or resistance, enrollment and retention 
rates, facility and usage, motivation, morale, complaints, suggestions, evidence of confidence 
building, initiative, perceived skill changes, computer intimidation, and instructor 
performance. 

This on-going monitoring and feedback helped CertainTeed to identify several key problem 
areas in its program. However, CertainTeed was not able to make major changes mid- 
stream in its program operations. One of the major issues CertainTeed identified early on in 
its program operations was the impact of making the program voluntary. Simply stated, a 
voluntary program does not usually reach those workers who most need training. In 
addition, although CertainTeed's participation rates were extremely high for a voluntary 
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program, the program did not achieve the "critical mass" of participation needed to ensure 
the most significant impact possible on overall productivity. 



CertainTeed assessed training program resources, learning processes and methods, and 
delivery systems on an informal but on-going basis to help ensure program quality. Principal 
achievements and problems were identified by focusing on elements such as scope and 
relevancy of curriculum, appropriateness of training levels, accessibility, incentives and 
disincentives, protection of participant confidentiality and privacy, roles and actions of 
project partners and suppliers, supportive learning environment issues, reactions to evaluation 
plans and processes, and administrative functions and concerns. 

The quality of over-all operations was looked at with an eye toward the capacity of the 
training program to meet both business and training needs as planned. Key questions asked 
were: 

• Is the training program working as planned? 

• How does the program compare with other exemplary workplace literacy programs? 

• Can the Precision Strike training program help achieve and sustain roofing products 
group business objectives? 

• What changes need to be made to improve program operations and results? 
Step 2: Promote the program 

Your precision strike training program needs to remain visible in the eyes of potential 
participants and senior management if it is to be successful. Potential participants need to be 
kept aware of the program and the benefits it can provide so that they continue to enroll. 
Senior managers need to be reminded of the program's contributions to the company's 
bottom line or they might cut the program during a budget review. 

CertainTeed periodically publishes a program newsletter to advise and update participants on 
issues related to the program. The newsletter is developed by the Training Supervisor and 
other program staff. The newsletter primarily serves as a communication tool, but is also 
used to recognize participants and their efforts. 

CertainTeed has always looked beyond potential participants in its promotion campaigns. On 
the day of CertainTeed' s grand opening, the training center was opened to the public and the 
entire community. Local dignitaries, including North Carolina Congressman Tim Valentine, 
participated in a formal ribbon cutting ceremony and tours of the CertainTeed plant and 
training center. 
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Numerous civic groups, individuals, and representatives from economic development groups 
and other industries have been oriented to the training program and have toured the center. 
The local newspaper has written several very favorable articles and editorials about the 
program. 
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IV, EVALUATION 



TASK 1: FORMULATE YOUR RESEARCH DESIGN 

A strong evaluation component is essential to any training program. Without an evaluation, 
you will not be able to determine whether the benefits of your precision strike training 
program exceed its costs. The overriding goal of your evaluation should be to determine the 
impact of your training program on such things as employees' work behavior, manufacturing 
productivity, and product quality, as well as on employees' basic and functional skills. 

The following steps should be taken in formulating your research design: 

• Define your research questions. 

• Select your research approach. 

• Select your specific design methodology. 

• Determine how to apply the design to your manufacturing plant. 

• Define your training group and your comparison group, 

• Determine the non-training variables that can drive your production costs. 

• Select appropriate statistical techniques for analyzing your data. 

Step 1: Define your research questions 

Defining your research questions is an essential first step to formulating your research 
design. These questions should clearly indicate what you want the evaluation to determine.. 

There are two general kinds of evaluations: formative or process, and summative or 
outcome. Process evaluations ask such questions as how the training was planned and 
operated, who participated, and where and when the training was conducted. The purpose of 
this type of evaluation is to determine whether the training ran smoothly and whether ana 
how operations could be improved. 

In the case of an outcome evaluation, the focus of this guidebook, you should be concerned 
with such questions as: 

• Has learning taken place? 
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Has there been a measurable increase in productivity? 



• Is the effort cost effective? 

In order to visualize how the research questions will be measured, it is helpful to phrase 
them as hypotheses. For example, CertainTeed assumed that: 

• The Precision Strike crew would score higher on a standardized test after the training 
than before. 

• The Precision Strike crew's run rate (roofing squares produced per hour) would be 
higher after the training than before the training. 

• Increases in the run rate, reductions in waste, and qurJker problem solving would 
produce savings that would more than compensate for the cost of the training. 

If you plan to conduct a rigorous evaluation and test any behavioral or productivity changes 
you found for "statistical significance," you will have to state these hypotheses as "null 
hypotheses" — hypotheses that assert there will be no change attributable to the training - 
and then seek to disprove these hypotheses. 

Several null hypotheses were tested statistically by CertainTeed, including: 

• There will be no statistically significant difference between the learning gains of 
Precision Strike program participants and those of non-participants. 

• There will be no statistically significant difference between "cost driver" measures 
(e.g. process inefficiency, waste, delay, run rate, short term complaints, and 
consistency and accuracy of filler percentage and bundle weights) prior to the 
Precision Strike program and such measures after the Precision Strike program. 

Step 2: Select your research approach 

Once you have formulated your research questions and hypotheses, you should select the 
variables, or measures, you want to study. These measures constitute your research 
approach or an evaluation model. 

CertainTeed' s evaluation model was designed to measure both learning gains and productivity 
gains. CertainTeed knew that if there were both a demonstrable level of learning attributable 
to the Precision Strike program and statistically significant positive changes in the company's 
"cost-drivers" and/or human resource measures, a connection would be established between 
investments in training and the return on those investments. 
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CERTAINTEED'S EVALUATION MODEL 


To Demonstrate Learning Gains - 


1. 


Comparison of employees' Test of Adult Basic Education (TABE) scores prior to 
the opening of the training center with their scores after the center has been open 
18 months. 


2. 


Longitudinal analysis of the training center's usage statistics (e.g., training 
modules mastered, number of student instructional hours, grade level increases by 
subject matter, etc.). 


To Demonstrate Productivity Gains - 


I. 


Change over time in selected quantifiable human resources measures, such as 
attendance, turnover, OSHA-recordable accidents, or participation on continuous 
improvement teams. 


2. 


Comparison of the results of a survey of plant workers' capabilities before the 
training center opens, and after 6 months, 12 months, and 18 months of operation. 


3. 


Comparison of average employee performance appraisal ratings for an eighteen 
month period before the training center opens with the average ratings during the 
eighteen months following its opening. 


4. 


Change over time in five key CertainTeed roofing products group "cost-drivers" 
(data to be maintained by line, crew, and product): 

- scrap as a percent of finished product, 

- down time as a percent of scheduled production time, 

- average run rate in roofing squares per hour, 

- change in the standard deviation of filler percent, and 

- change in the standard deviation of finished bundle weight. 



Step 3: Select your specific research methodology 



After you have your evaluation model, you need to consider the kind of formal methodology 
you will use. Approaches, described briefly here, include: 

• Correlational research investigates the extent to which variations in one variable or 
measure correspond with variations in one or more other variables. 
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• Causal-comparative research investigates possible cause and effect relationships by 
observing some existing consequence and searching back through the data for 
plausible causal factors. The limitation of causal-comparative research is that it is 
difficult to prove cause and effect. In most cases, all that can be concluded is that a 
relationship between variables exists. 

• Quasi-experimental research approximates the true experimental conditions that exist 
when the researcher is able to both control and manipulate all of the variables 
pertinent to the study. In quasi-experimental research, both a participating and a non- 
participating (control) group receive pre- and post-tests, and the changes in scores of 
the two groups are compared. 

The quasi-experimental design may not adequately control for all of the factors other than the 
precision strike training that could affect learning and productivity gains. When the design is 
unable to control for alternative explanations, other than the training program, for changes 
that occur, a threat to validity of the study exists. There are two validity issues, internal and 
external. 

• Internal validity asks the question: Did, in fact, the experimental treatments make a 
difference in this specific instance? 

• External validity asks the question: To what populations, settings, treatment 
variables, and measurement variables can this effect be generalized? 

CertainTeed gave a great deal of attention to identifying the extraneous variables that might 
affect results, and controlling for them as much as possible. These are discussed in the box 
below. 
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VARIABLES THAT MAY AFFECT INTERNAL AND EXTERNAL VALIDITY 



Internal Validity 

1. History — specific events, in addition to the training, occurring between the first 
and second measurement of effects. 

2. Maturation — changes in the participants that result from the passage of time per 
se (e.g. growing older, hungrier, fatigued, or less attentive). 

3. Testing — the effects of testing upon the scores of a subsequent testing. 

4. Instrumentation — changes in measurement due to changes in instrument 
calibration or changes in the observers or judges. 

5 . Statistical Regression — a phenomenon occurring when groups have been selected 
on the basis of extreme scores. 

6. Selection - biases resulting from differences in those selected for the participating 
and non-participating groups. 

7. Experimental Mortality — differences between participating and non-participating 
groups in the loss of participants. 

8. Selection-Maturation Interaction — effects from interaction among any of the 
above variables which can be mistaken for the effects of the training. 

External Validity (Representativeness) 

1 . Interaction of Selection and Training — effects from the interaction between 
selection biases and training. 

2. Reactive or Interaction Effect of Pre-testing — changes in participants due to pre- 
testing that modify the way that they respond to training. 

3. Reactive Effects of Experimental Procedures — effects arising from the 
experimental setting which will not occur in non-experimental settings. 

4. Multiple-Treatment Interference - effects from prior treatments (training) 
influencing subsequent treatments. 

Source: Donald T. Campbell and J. C. Stanley, Experimental and Quasi-experimental 
Designs for Research, Chicago: Rand McNally and Company, 1966, 
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In planning your research methodology, you also need to define the scope and time frame of 
your investigations. Ideally, your evaluation should begin when individuals are first assessed 
for your precision strike training since pre-tests of individuals in both the participating and 
non-participating groups are a vital part of the evaluation. For this to occur, you need to 
begin thinking about and structuring your evaluation at the same time that you are laying the 
groundwork for the training program. 

CertainTeed's evaluation model, which included a quasi-experimental design, was developed 
during the initial research and analysis phase, and then further refined during program 
implementation. 

Step 4: Determine how to apply the design to your manufacturing plant 

In conducting the evaluation in an industrial setting, you will want to ensure that your design 
is both unobtrusive and non- threatening. It must not be detrimental to the primary function 
of your business — producing goods and/or services. A design that disrupted your plant's 
economic activity would be counterproductive. 

Some of the limitations CertainTeed imposed on the evaluation were: 

• Restrictions on altering work roles; 

• Prohibition against using random sampling to assign individuals to either the 
participating or non-participating group; 

• Discouragement of withholding training from all those who wanted it; 

• Limitations on the amount of time allotted to conduct the evaluation; and 

• Constraints on disrupting work activities. 

These limitations led CertainTeed to select a quasi-experimental research design. When used 
in a business context, this design can more accurately mirror actual industrial conditions and 
reflect the populations under study than true random experiments or laboratory settings 
could. In other words, the quasi-experimental design offers the needed controls for 
acceptable research results, while at the same time being more practical to employ. 

When applying the design in your manufacturing plant, you need to "tighten up the design." 
This involves reducing invalidity and controlling measurement error. Invalidity can be 
reduced by improving the comparability between individuals in the participating and non- 
participating groups, and repeating the evaluation a number of times to strengthen the extent 
to which training effects can be generalized. 
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Measurement error can be reduced either by ensuring that "training" is the same for all 
individuals, or by taking into account any differences in this training. For example, in 
considering training to be a "unitary variable" at CertainTeed, the evaluators had to take into 
consideration the effects of differences in instructors' preparation and skills, teaching 
materials, teaching aids, and individual tutoring. Precision strike training can only be treated 
as a unitary variable if: 

• All of the elements that affect training converge together to affect student 
performance; and 

• The differences in training fall equally on individuals in the participating and non- 
participating groups. 

If you choose a quasi-experimental design, you will want to use variations most suited to 
your manufacturing plant. CertainTeed chose two variations, displayed graphically in the 
box below. The first involved conducting one pre-test (Tl) of individuals in the 
experimental (participating) and control (non-participating) groups, providing training (X) 
only for those individuals in the experimental group, conducting one post-test (T2) of 
individuals in each group, measuring the change in test scores for each individual, and theji 
comparing the average change in test scores between the two groups. 

In the second, two pre-tests (Tl and T2) and two post-tests (T3 and T4) were given to 
individuals in each group and the change in the average score for each individual was 
calculated. Finally, a comparison was made between the average change in average test 
scores between the two groups. 
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VARIATIONS OF QUASI-EXPERIMENTAL DESIGNS USED BY CERTAINTEED 


Non-randomized Control Group Pre-test/Post-test Design 




Pre-test Treatment 


Post-test 


Experimental 


Tl X 


T2 


Control 


Tl 
11 


1 Z 




Control Group Time Series Design 






Pre-test Treatment 


Post-test 


Experimental 


Tl T2 X 


T3 T4 


Control 


Tl T2 


T3 T4 



Step 5: Define your training group and comparison group 



If you choose to use a quasi-experimental design, you will have to decide which workers will 
receive the precision strike training and which workers will constitute the comparison group. 
As has been discussed earlier, you may need to offer incentives to the training group, 
particularly if some of the training occurs on their own time. You may also have to offer 
assurances to the comparison group that they will have an opportunity to participate in the 
training if it proves successful. 

Concerns about internal validity should be kept in mind when selecting the treatment and 
comparison groups. 

• Participants and non-participants should be selected in such a way that they represent 
the population being studied; and 

• Every attempt should be made to limit events or actions that might affect participants 
and non-participants and confuse the measurement of changes in the two groups 
resulting from training. 

Step 6: Determine the non-training variables that can drive your production costs 

All manufacturing plants have some non-training variables that can drive up production costs. 
Some of these costs are directly related to the manufacturing process itself, while others 
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relate to human resources. Variables directly related to the manufacturing process include 
run rate, waste, delay, and operational delay. Variables related to human resources include 
attendance and turnover. 

In conducting your evaluation, it is important that you first identify and then measure these 
variables before, during, and after your training program. In this way you can determine 
whether any changes in learning and productivity you find might be attributable to changes in 
these variables, and not solely to your training program. 

Step 7: Select appropriate statistical techniques for analyzing your data 

As was mentioned earlier, if you want to conduct a rigorous evaluation you will have to test 
any learning or productivity changes you find for "statistical significance." For a quasi- 
experimental design evaluation, tests for statistical significance involve determining whether 
differences in the changes you find between the experimental and control groups could be the 
result of chance, or whether they are attributable to the training. There are a number of 
statistical tests that can be done to make this determination. If you are interested in pursuing 
them, you should consult textbooks in statistics and program evaluation. 
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TASK 2: DEFINE THE DATA COLLECTION TOOLS 



An evaluation of a precision strike training program requires multiple data collection tools so 
that you can measure both learning gains of individuals and productivity measures such as 
run rates. The data collection tools, which are discussed in this task, range from 
standardized tests, to computerized productivity data on the "cost driver" variables, to 
surveys of trainees and supervisors. 

Step 1: Select standardized instruments 

There are not many standardized tests that are appropriate for the workplace. The Tests of 
Applied Literacy Skills, made available after the inception of CertainTeed's literacy program, 
have three sections: prose, document, and quantitative. Some of the questions in these tests 
would be applicable to the workplace, although none are especially relevant to a 
manufacturing environment. 

The other widely used test in adult literacy programs is the Test of Adult Basic Education 
(TABE). This test ranks individuals by grade level rather than by workplace competencies. 

Either of these tests can be used as the pre- and post-tests given to both the precision strike 
and the comparison groups. However, you should remember that the TABE is not a good 
measure of participants' learning gains in workplace literacy skills; it is only designed to 
measure grade-level classroom skills. 

Most workplace literacy experts and practitioners recognize the limitations of the use of 
standardized tests such as the TABE. However, standardized tests can provide useful 
information if they are used as a part of a mix of custom-designed assessments based on 
workplace materials and activities. 

Step 2: Construct a Cloze Test 

The Cloze test, discussed in section I under research and analysis, can also be used as pre- 
and post-tests. This test can be constructed fairly simply. You should first select work- 
related material that is at the level of difficulty generally encountered by employees, delete 
words from the passage, have those bfeing tested fill in the deleted words, count the correct 
number of responses, and then calculate the percentage of correct responses. 

A scale was used to organize Cloze test responses at CertainTeed. A 70 percent or higher 
accuracy rate in responses was equivalent to a score of 90 percent on other types of reading 
tests. Scores in this range indicated an employee who could easily comprehend materials at 
this level independently. 

A 43-65 percent accuracy rate indicated an employee who could successfully comprehend 
materials at this level with some instructor assistance. 
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Scores of below 43 percent accuracy indicated an employee who would have difficulty 
comprehending similar materials and would become frustrated without considerable help, 
support, and/or prior remedial basic skills instruction. 

Step 3: Design employee and supervisor surveys 

The purpose of employee self-assessment surveys is to compare employees' perceptions of 
how effective they are in performing a set of work-related tasks before and after the 
precision strike training. In CertainTeed's self-assessment, employees were asked to 
indicate, on a five point scale ranging from "never" to "always," their degree of agreement 
with a series of statements such as "I have trouble reading the manual used for my job." 

The questions on CertainTeed's survey, a modification of a survey designed by an external 
vendor, measured employee perceptions of the following skill areas: 

Goal orientation 
Problem solving skills 
Analytical skills 
Effectiveness 
Personal competence 
Process improvement team skills 
Reading skills 
Math skills 
Writing skills 
Technical skills 
Responsiveness 
Attention to detail 
Monitoring skills 
Learning readiness 

The employee self-assessment survey can be augmented by supervisor rating surveys 
measuring their perceptions of employees' work skills. In these surveys, employees are 
rated as a group, not as individuals. The substance of the questions and the skill areas 
measured would be the same as those for the employee survey. 

Step 4: Design structured interviews 

In addition to written employee and employer surveys, you should consider conducting 
structured interviews. T wo types were conducted as part of the CertainTeed evaluation. 

The first, longitudinal interviews, were designed to elicit feedback from both training 
participants and non-participants on how they felt about the Precision Strike training program 
and any ideas they had for improving it. 
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The second, post-program interviews, were designed to collect anecdotal information on any 
changes or impacts that individual participants experienced and/or supervisors observed. 



Step 5: Develop a system to collect other existing data 

You will probably find that your manufacturing plant already collects a wealth of data 
applicable to your program evaluation. However, you need to collect this data, and you may 
even have to convert it to another form in order to make it more useable in your evaluation. 
It is this data that is most likely to give you a fuller picture of the interaction between 
learning and productivity changes. 

At CertainTeed the following existing data were used: 

• Worker Attendance Records — defined as the number of hours an individual misses as 
a percent of the number of hours scheduled. 

• Accident Investigation Reports -- completed by a supervisor for every accident, and 
shows who was involved, and if the accident was a near miss, personal injury, or 
damage to equipment or material. 

• First Aid Reports — detailed by reason and by individual. 

• Performance Appraisal Forms — completed annually on each employee. 

• Employee Suggestion Program Forms and Records -- includes the names of individuals 
and teams making the suggestions, any monetary award amounts, the status of the 
suggestions, any actions taken on the suggestions, the reasons for the actions, and any 
impacts resulting from the actions. 

• Shipping Error Records. 

• Customer Complaint Records ~ includes both short-term and long-term complaint 
records. 

• Finished Product Inventory Audit Forms — completed as part of the company's on- 
going quality efforts. Bundles of shingles are randomly selected for inspection and 
subjected to a wide range of quality tests. 

• CertainTeed VAX Computer System Reports — includes numerous productivity and 
product quality measures. The "cost driver" variables extracted from this system and 
used for the CertainTeed evaluation were: 

- process inefficiency, 
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waste, 

waste attributable to buildups, sheet breaks, contamination, dimension problems, 
back coating, tape, etc., 

delay, 

delay attributable to operator error, 
delay attributable to operator problems, 
run rate, 

filler percentage, 

bundle weight, and 

process monitoring system audit data. 
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TASK 3: IMPLEMENT YOUR EVALUATION 



Once you have determined your evaluation model, defined your variables, and designed your 
data collection tools, you are ready to implement your evaluation. In implementing the 
evaluation, you should minimize disruption of the manufacturing process as much as 
possible. In order to do this, the data collection process should be carefully planned and 
coordinated with other activities at the plant. 

Step 1: Administer assessment tests and surveys 

Before you begin your precision strike training program you will want to administer any 
standardized tests you have selected, as welfas the Cloze test. The employee and supervisor 
surveys should also be administered at this time. These tests will be repeated at the end of 
the training period. 

At CertainTeed the TABE was one of the pre- and post-tests administered to both participants 
and non-participants in the Precision Strike training program. Employees were allowed to 
take the test "on the clock." The Cloze test was also administered to both participants and 
non-participants. You should consider using the Tests of Applied Literacy Skills for your 
pre- and post-tests. 

The Employee Self Assessment Surveys and the Supervisor Rating Surveys were completed 
before the start of Precision Strike training. They were administered again the month after 
Precision Strike training finished. 

Step 2: Conduct longitudinal and post-program interviews 

The structured interviews, as noted above, are primarily designed to receive program 
feedback. For the longitudinal interviews, a group of workers randomly selected before the 
start of the training (participants and non-participants) should be interviewed at the beginning 
and several times during the course of the training. The information obtained from these 
interviews should be provided as feedback to the program instructors and designers, as 
appropriate. Changes can then be made to improve the program as it progresses. 

You should randomly select participants, non-participants, and supervisors for the post- 
program interviews. The information obtained in the post-program interviews will provide a 
fuller picture about your precision strike training than could be provided by quantitative data 
alone. It might also serve to explain some unexpected quantitative results. 

Step 3: Compile and analyze other data collected 

If you have selected an integrated computer system as part of your precision strike training, 
the reports generated by this system should form an integral part of your evaluation. These 
data provide information on learner activities, progress, and results. 
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CertainTeed extracted data from the management program of its integrated computer system 
and used them to show such results as lessons assigned and mastered; training modules 
assigned and mastered; grade level gains in math, reading, and writing; and training levels 
completed. 

Other data that you might want to compile and analyze include: 

• The names and number of individuals who receive structured on-the-job training, and 
the number of their training hours; 

• The types of seminars or classes individuals attend, and the names of individuals who 
participate in and complete these activities; 

• The names of individual who complete training levels; 

• Records of various promotional and incentive activities; and 

• Data from precision strike program participants' individual file folders. 
Step 4: Compile and analyze the non-training data 

Non-training data that can give you a fuller picture of the productivity impacts of your 
precision strike training should also be collected. This type of data can help you determine 
whether there is a correlation between your precision strike training and changes in these 
productivity measures. 

Places to search for relevant data include attendance records, accident reports, first aid 
reports, and suggestion forms. For example, if attendance improved and accidents decreased 
for training participants but not for non-participants, you might infer that these changes 
resulted from the training and that the training had a positive effect on productivity. 

Performance appraisal data may also be compiled and analyzed, though you will have to use 
the utmost care to maintain confidentiality. At CertainTeed, employees are rated from a low 
of one to a high of six on their performance appraisals. CertainTeed' s evaluation compared 
pre- and post-training performance appraisal ratings to learning gains and changes in crew 
productivity to determine whether there was a correlation between the performance appraisals 
and learning gains and between the performance appraisals and changes in crew productivity. 

Shipping error records can also be used to measure productivity. Because CertainTeed's data 
base makes it possible to attribute shipping errors to particular individuals, CertainTeed's 
evaluation was able to compare changes in an individual's shipping error rate to that 
employee's learning gains. 
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Another source of data is customer complaints attributable to individuals in the precision 
strike program prior to and after training. Quality is, of course, part of any measure of 
productivity. If your company does not have a way to identify which individuals or crews 
are responsible for a mistake that leads to a customer complaint, you should consider 
implementing such a system. Initially, CertainTeed stenciled identifying codes on the wraps 
of bundles of shingles. Later, CertainTeed implemented a tape coding system that placed 
codes identifying when the product was produced and who produced it directly on the shingle 
itself. 

If your company has an overall measure of crew performance, this data could augment any 
insights gained from analyzing shipping errors and customer complaints. CertainTeed 
calculates summary crew performance ratings. These ratings are determined by a series of 
quality tests - weekly audits that are conducted randomly - on as many as 25 different 
items. 

You may also have process monitoring system audit data that can be useful in your 
evaluation. This system uses quality assurance staff to analyze the discrepancies between 
source data and actual data entered in the process monitoring system. At CertainTeed, 
records were kept of data that should be in the system but were not there, and data, dates, 
and times entered incorrectly. These data served as an excellent diagnostic tool for basic 
skills training and as a way to track individual operator competency. 

The cost-driver variables mentioned previously can play a key role in your precision strike 
evaluation. You will want to compare the precision strike training crew and the comparison 
group on these productivity variables for a several month period prior to the training and the 
same length period after the training. Differences that take place only in the precision strike 
crew may be assumed to be the result of the training, as long as you have successfully 
maximized internal and external validity. 

Step 5: Determine the statistical validity of your results 

In Step 4, you began to look at correlations between training, learning gains, and 
productivity changes. To be absolutely sure that these correlations are "statistically 
significant" - that is, that they are not the result of chance - you need to conduct 
sophisticated statistical tests. Again, you should consult statistical or program evaluation 
textbooks to learn the methods for making these determinations. 
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APPENDIX I: WORKPLACE LITEk \CY TOOLS 



THE NOMINAL GROUP TECHNIQUE 

Description 

Introduction 

In response to the need for more effective group problem-solving methods, Van de Ven and 
Delbecq created the nominal group technique (NGT). This technique is a structured activity 
that provides an orderly procedure for systematically obtaining qualitative information from a 
group and for establishing priorities for that information. The word "nominal" was 
previously used to refer to individuals who were prohibited verbal interaction. When using 
this technique, individuals are only "nominally" a group (hence the technique's name). 
Individuals work in the presence of others but don't interact verbally except at specified 
times and in limited ways. 

Format and Procedure 

In order to counteract inhibiting influences found in the interacting group process, Delbecq 
and Van de Ven developed NGT along the following format: 

Imagine a meeting room in which seven to ten individuals are sitting around a 
table in full view of each other; however, at the beginning of the meeting they 
do not speak to each other. Instead, each individual is writing ideas on a pad 
of paper in front of him or her. At the end of five to ten minutes, a structured 
sharing of ideas takes place. Each individual, in round-robin fashion, presents 
one idea from his or her private list. A recorder writes that idea on a flip 
chart in full view of other members. There is still no discussion at this point 
of the meeting — only the recording of privately narrated ideas. Round-robin 
listing continues until all members indicate they have no further ideas to share. 
A highly structured discussion follows in which each idea is reviewed for the 
purpose of clarification. Statements of agreement and disagreement may be 
expressed a this point. The discussion is structured in a manner that allows 
each idea to receive attention. After the discussion comes voting. The group 
priorities are the mathematically pooled outcomes of individual votes. 

In summary, the NGT decision-making process involves the following four stages: 

1) Silent generation of ideas in writing. 

2) Round-robin feedback from group members to record each idea in a terse phrase on a 
flip chart. 
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3) 



Discussion of each recorded idea. 



4) Group priorities mathematically derived through rank-ordering or rating. 
Need 

Observation of normal interacting groups reveals that they tend to fasten on that aspect of a 
problem which appears early on in their discussions (and this is often a relatively superficial 
aspect). They're apt to focus on one particular train of thought. We're solution-minded in 
our culture, and when there's conflict, or a group is threatened, things get even worse. Then 
"group think" — follow-the-leader style thinking - tends to set in. There's a feeling that one 
should get down to solving the problem without dragging out the business of defining and 
dissecting it. And, apparently, the greater the crisis, the greater the pressure to move 
quickly to solve it. 

Besides, most groups contain individuals who already have "the answer" — some pet idea or 
other ("Computerize!" "Decentralize!" "Coordinate and Integrate!" "Reorganize!"). These 
individuals are, in effect, solutions going around in search of problems. Also, groups tend to 
undervalue ideas, whatever their substance, if those ideas aren't presented with facility and 
vigor — or if the ideas are put forward by a low-status member of the group. And, 
normally, a lot of time and energy is wasted on group maintenance — smothering verbal fist 
fights or patching up the ill feelings left behind after the group pecking order has been 
affirmed. As result of all of the above, there's usually a very high rate of kill-off of ideas 
that are at their formative stages. 

The nominal group procedure is designed to prevent dominance games: normally, it prevents 
domination of the group by individuals. It ends group pressure for conformity, and it 
eliminates the inhibitions imposed by the group's pecking order on the lowly members of the 
group. The procedure also prevents "premature closure" — making up your mind before all 
the data are in. Findings show that group decisions made after full review of data are 
significantly better than those made before such a review. Also, it appears that if you're 
working in the presence of others who are supportive of what you're doing, you tend to be 
more productive than when you're working on your own. 

Characteristics 

The nominal group technique has been found superior to conventional discussion groups 
when applied to problems concerned with judgmental decision making. The following 
characteristics of NGT account for its advantages over other interacting groups. Nominal 
groups: 

1) Stimulate creative tension by means of the presence of others, the silence, and the 
evidence of activity. This tension is important for individual commitment to the 
search process. Thus, the social facilitation of the group setting is retained and 
amplified. 
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2) Avoid evaluation or elaborating comments while problem dimensions are being 
generated. 

3) Provide each individual time and opportunity to engage in reflection (search) and 
force participants to record their thoughts. 

4) Avoid the dominance of group output by strong personality types. 

5) Prevent premature closure to the alternative search process and decision making. 

6) Allow all participants to share in the opportunity for influencing the direction of group 
decision outcomes. 



7) Encourage the generation of minority opinions and ideas, which consequently are 
more likely to be voiced. 

8) Tolerate conflicting, incompatible ideas since all ideas are revealed in writing. 

9) Alleviate "hidden agendas" or covert political group dynamics ~ these rarely develop 
during writing exercises. 



10) Induce a sense of responsibility in the members to achieve group success. 

11) Impose a burden upon all participants to work and produce their share in the 
necessary task. 

12) Induce a greater feeling of commitment and a greater sense of permanence through 
written expression. 

Benefits 

Interestingly, NGT was originally developed, initially tested, and first applied in Office of 
Economic Opportunity community action programs to facilitate the involvement of 
disadvantaged citizens. Its tremendous success and consequent extensive use in encouraging 
consumer involvement can be attributed to the NGT characteristics previously cited, 
specifically numbers 2, 4, 6, 7, 8, and 9. Participant satisfaction with the NGT process 
remains high because it is often the individual's first experience in which verbal glibriess and 
fluency is not the overriding base of power. 

These same characteristics of NGT are responsible for its successful ability to balance 
participation among verbal organizational members, especially when their placement or 
function in the hierarchical system may account for significant differences and negative 
influences of power, authority, and status. The balanced concern for task accomplishment 
and interpersonal social maintenance functions accompanied by equality of participation 
serves to facilitate problem solving and increases participants' satisfaction with the process. 
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In summary, the effectiveness of NGT as opposed to traditional methods is based on four 
types of evidence: 

1) The greater flow of ideas; 

2) The fuller participation of all those involved; 

3) The ^creased evidence of the task being brought to closure; and 

4) The evident sense of satisfaction on the part of participants. 

Used in the appropriate situation, NGT is faster and less expensive, involves more people in 
a shorter period of time than other approaches, and creates a climate where communications 
are open and above board and participants have the opportunity to pin-point priority items 
that need immediate attention. 

Drawbacks 

Associated with the positive characteristics of NGT are difficulties found in conducting some 
meetings. Extended preparation is required for nominal group meetings in order to clearly 
identify the information desired from the group and to formulate the NGT question which is 
presented to the group at the beginning of the session. NGT, therefore, is not a spontaneous 
group meeting technique. 

The inflexibility of the structured NGT format makes it hard to make adjustments or to 
change topics in the middle of a meeting. NGT is generally limited to a single-purpose, 
single-topic meeting. 

Conforming behavior to a structured format is required on the part of all participants. This 
may be uncomfortable for inexperienced participants or cause resentment from the group 
"heavies" who prefer to hog the limelight before a spectating audience. 

NGT has also been found to be exhilarating but exhausting because of its requirement for 
active participation without a let-up. But often the best ideas come just when people are 
fairly exhausted, towards the end of the round-robin stage, or in the discussion that follows. 

Usage 

Nominal groups are useful for placing primary emphasis on the task and getting simultaneous 
efforts and information on various aspects of a probiem. The most typical applications in 
which the NGT process has been determined most effective include the identification of 
problems, the exploration of solutions, and the setting of priorities. Thus, NGT is effective 
for the generation, assimilation, and exchange of qualitative information. 
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Qualitative Information 



Qualitative input into decision-making processes is needed for a variety of reasons. First and 
foremost, the emotional, attitudinal, and interpersonal factors of plant employees are 
important in understanding the workers' literacy problems and workplace needs. This type 
of information is not easily quantifiable. Second, political variables and vested interests in 
operating change and improvements are important. Quantification of these various political 
and emotional considerations is exceedingly difficult. Third, specialization in a plant 
produces language barriers between individuals. Quantitative data and certain analytical 
models within a particular specialized unit may not be easily understood, communicated, or 
utilized by others. Finally, quantitative data often cannot be validly or reliably produced 
until qualitative understanding of the problem first exists. Because of these critical needs for 
qualitative information, NGT may be used essentially as a process for generating qualitative 
insigh. in regard to the critical dimensions of a particular problem, such as literacy skills in 
the workplace. 

Judgmental Problem Solving 

Utilized as an instrument to improve decision-making, the nominal group has many varied 
and valuable uses. First, it may serve as a problem prevention mechanism by expanding the 
dimensions of problem identification, thus preventing error and unwarranted risks. Because 
of the ease and quickness with which the nominal group may be created, timely 
considerations may be obtained in order to help anticipate major problems and confront them 
in earlier stages. The ability of the nominal group to react quickly and thus more effectively 
and to react with rapid and intense involvement constitutes a major advantage. Additionally, 
the nominal group may help increase a more penetrating analysis of present and potential 
literacy problems. 

The nominal group may increase the adequacy of the search for alternatives by a) increasing 
the scope and/or depth of alternative courses of action, and b) accessing more adequately the 
utility functions of such alternative solutions. Increasing the number of alternatives identified 
may be desirable in broadening the courses of action under consideration. Accessing the 
quality of alternatives is provided for by the ranking and voting procedures of the NGT 
process. In addition, the nominal group may enhance the evaluation of alternative solutions 
by providing for the identification of side effects or the unintended consequences of certain 
operating procedures and/or processes. Using a variety of employees in the nominal group 
may be particularly effective in this situation, since the consequences of alternatives are 
usually quite ambiguous and hard to predict. 

Poor decisions are often due to the lack of sufficient information or the inability to foresee 
consequences. The nominal group may be most appropriate and effective at plants where 
there is a lack of agreement or incomplete state of knowledge concerning the nature of the 
literacy problem or components of the problem. Since uncertain and ambiguous situations 
are often the norm in certain plant operations, the nominal group lends tremendous potential 
for improved decisions in regard to literacy skills needs. 
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Communication Channel 

The nominal group may be used to obtain information input and feedback in plant operations. 
It offers further advantages in counteracting the communication dysfunctions inherent in 
differentiated plant jobs and functions. 

The nominal group may be created to assure that the correct people get the correct 
information at the correct time. Used for this purpose, the nominal group would become a 
communication channel to transmit appropriate communication to the appropriate decision- 
making center. A simple increase in communication does not necessarily constitute 
improvement. The great advantage of the nominal group in this situation is that it offers the 
needed restricted communication channel by controlling a) the selection of information 
needed through use of the specific nominal questions, and b) the nature of the communication 
process through use of the structured NGT process. Thus the specificity of the NGT 
structure implies a selectivity of both communication channels and the information 
communicated. 

In addition to the use of the nominal group in obtaining appropriate information on literacy 
skills, the nominal group may also be created to open up previously closed circuits of 
communications. The nominal group can thus encourage the acquisition of presently absent 
information which is closely tied to on-going job functions or might even be a critical and 
integral part of plant operations. Use of the nominal group as a communications channel 
adds the needed dynamic dimension to plant functioning and introduces flexibility and 
responsiveness into an otherwise rigid and static system. 

Training Program Planning 

Because the nominal group is effective in situations that are uncertain and/or ambiguous, and 
because planning for workplace literacy skill programs is complex and usually has many 
uncertain elements, NGT can be a useful tool. Administrators, trainers, planners, and other 
staff are increasingly being asked to participate in special committees, task forces, or 
technical teams concerned with recommending new training programs or changing existing 
programs. Planning program training often can be uncertain and complex because of limited 
awareness of the importance of the workers' actual problems at which the training is aimed, 
and because of limited understanding of available solutions or "effective" educational 
programs to model. Lack of empirically validated plant operations technology and methods 

compounds the problem. The nominal group can be effective in identifying a wide range of 
data essential to effective training program planning for workplace literacy skills. 

NGT is also appropriate because a large number of individuals or groups may have input into 
plant operations. The nominal group may be useful in eliciting data from heterogenous 
groups with different value systems, conceptual orientations, and job functions. The nominal 
group can also be utilized for establishing the setting for joint participation in planning 
literacy skills training programs. 
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In summary, the nominal group can be effectively used to: 



• Involve administrators and a variety of employees in early review of training needs 
and program plans, and explore problems and document unmet training needs. 

• Involve outside resource people and specialists in exploring appropriate programs to 
solve these needs. 

• Involve appropriate staff in developing designs for implementation and evaluation of 
training programs. 

• Involve personnel from elsewhere who may be later adopters of the training program 
or who will be closely involved in training program implementation and/or delivery. 
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CERTAINTEED'S INTERVIEW INSTRUMENT FOR DETERMINING LITERACY 
SKILLS 



I would like to get examples of times during the la^t month or so when you used printed 
materials in connection with carrying out some part of your job. Could you give me the 
exact name of this material(s)? 

NAME OF MATERIAL(S) 

(Code) 

A. Total book 

B. Part of book (text) 

C. Part of book (charts, tables, diagrams, map) 

D. Single to triple sheet text 

E. Single to triple sheet graphics 

How did you use that material in getting the information you needed? 
Why did you choose to read that particular material? 

Did you learn something from this material? How did you learn the material? (Use the 
information from the above question to code the following:) 

READING TO LEARN 

A. Reread/rehearse 

B. Problem solve/question 

C. Relate/associate 

D. Focus attention 

READING TO DO (no learning) 

E. Fact-finding in text 

F. Fact-finding in charts, graphs, tables, maps 

G. Following directions using text 

H. Following directions using charts, etc. 

READING TO DO WITH LEARNING 

I. Special learning strategy (explain) 
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READING TO ASSESS 

J. Usefulness for a particular task 

K. Whether to read more carefully later 

L. Whether to pass material on to someone else 

M. Other (specify) 

If you had to do the same work/ task tomorrow, would you have to read this material again? 
How important to the completion of the job was reading this material? 
How often do you do this kind of reading? 

JOB READING AND WRITING CHECK LIST (record yes or no answers). 
In vour work, do vou read Yes No 

1. Notes, letters or memos? — 

2. Forms (such as work orders, vouchers, 

claims, etc.)? — — 

3. Charts/graphs? _ _ 

4. Policy manuals, regulations, and 

instructions? — — 

Do vou use information from books such as 

1 . Telephone directories? — 

2. Catalogs? _ _ 

3. Dictionaries? — 

4. Technical references? 

5 . Company manuals? — 
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In your work, do you write 

1. Notes, letters, memos? 

2. Forms? 

3. Reports for superiors or others in 
your field? 

In your reading at work, do you have to 

1 . Use directions? 

2. Find out facts? 

3. Find out opinions, purposes or hidden 
meanings? 

4. Use two or more books at a time to 
find out information? 

5. Compare references from two or more 
books and make a value judgment on the 
one to use? 

In your written work at work, do you have to 

1 . Report on what was accomplished? 

2. Generate plans for further work? 

3. State your opinions about some aspects 
of the job? 

4. Complete already prepared forms? 



Now I'd like to get some examples of writing you have to do, Could you give me some 
instances when you had to write something on your job in the last month or so? 

What exactly did the writing task involve? 

TYPE OF TASK 

A. Fill out form 

B. Write letter, memo 

C. Write report or articles for others 

D. Note work accomplished 

E. Other (specify) 

What would be the consequences if you made a mistake writing this material? 
How often do you do this type of writing? 

[The interviewer should ask whatever other questions necessary to clarify the job analysis.] 



JOB RELATED LITERACY MATERIALS CHECK LIST 



Task Oriented Reading 



reference manuals 
measurement tools 
blueprints 



work orders, forms 
computer printouts 



diagrams & flowcharts 



job aids 

graphs, scales, charts, tables 
procedural guides 
first aid instructions 
metric conversion charts 
product labels 



General Job Reading 



safety manual 
insurance manual 
accident forms 
bulletin board memos 
payroll check stubs 
posters 



company policy manual 
insurance forms 
company newsletters 



union brochures 
training manuals 
textbooks 
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THE CLOZE PROCEDURE 



Description 

The Cloze test is an inexpensive, easy to administer measure of the ability of a reader to 
understand the sentences of a passage at a literal or factual level. It is often used in 
workforce literacy training to assess the reading level of job incumbents. 

A Cloze test can be constructed at any reading level desired. Cloze tests are constructed by 
selecting a reading passage, conducting a readability assessment to determine its difficulty 
level, and deleting words from the passage for the learner to fill in. 

The results of the Cloze tests and the readability assessment information serve as a guide in 
deciding upon the complexity of instructional materials. The Cloze test scores translate into 
ability categories that indicate the ease or difficulty learners will have in reading and using 
materials prepared at a certain difficulty level. In addition, the Cloze test can be 
administered again at the end of a project. The differences between the pre- and post-Cloze 
test scores serve as one means or assessing the effectiveness of the instruction provided. 

The Cloze test for each job was developed by the project consultant. Passages were selected 
from the new technical training manuals that were developed for each job. The tests were 
administered by the Vance-Granville Community College instructor assigned to CertainTeed. 
The tests were interpreted by the project consultant. Each individual on the crew was 
provided feedback on his or her test performance by the project consultant. 

Test Scoring Information 

Performance on Cloze tests correlates closely with performance on standardized reading tests 
at approximately r=.80 - .90, or a reliability of 80 - 90 percent. 

Tests are scored by simply counting the correct number of responses and calculating the 
percentage of correct responses. A scale was used to organize Cloze test results at 
CertainTeed. A 70 percent or higher accuracy rate in responses was equivalent to a score of 
90 percent correct on other types of reading tests. Scores in this range on a Cloze passage 
written at the 9+ level indicated an employee with advanced level literacy skills who could 
easily comprehend materials at this level independently, or without assistance of an 
instructor. 

A 43 to 65 percent accuracy rate in responses indicated an employee with intermediate-level 
literacy skills who could successfully comprehend materials at this level with some instructor 
assistance. 
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Scores of below 43 percent accuracy rate in responses indicated an employee who would 
have difficulty comprehending 9+ materials and would become frustrated without 
considerable help, support, and/or prerequisite remedial basic skill instruction. 

These three levels can be called Ability Levels, with the following designated scoring ranges: 

• Independent Level - 70 % - 100 % 

• Instructional Level - 43 % - 65 % 

• Frustrational Level - Below 43 % 
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TEST OF ADULT BASIC EDUCATION 



Description 

The Test of Adult Basic Education (TABE) is a set of norm-referenced tests designed to 
measure achievements in reading, mathematics, language, and spelling. The tests are 
designed to enable teachers and administrators to diagnose, evaluate, and successfully place 
examinees in adult education programs. In addition, because TABE scores and scores on the 
General Educational Development (GED) test are highly correlated, scores obtained on the 
TABE provide a means for predicting scores on the GED tests. Other uses of the TABE 
include pre- and post-testing to measure growth, and program evaluations. 

There are four overlapping levels and two parallel forms, forms 5 and 6, offered at each 
level. The levels and estimated grade ranges are as follows: 

Level Range 



Items are based on educational objectives and broad process classifications. The content 
categories were defined by examining current adult education curriculum guides, published 
texts, and instructional programs. The process classifications, including recall and 
recognition, inference, and evaluation, were derived from various taxonomies. 

A brief description of the content covered in each test follows: 

Reading Vocabulary. Test 1 contains 30 items that measure same meaning words, opposite 
meaning words, multi-meaning words, the meanings of affixes, and words in context. 

Reading Comprehension. Test 2 contains 40 items that measure comprehension of reading 
passages. Items test ability to extract details, analyze characters, identify main ideas, and 
interpret events described in passages. Items also test ability to differentiate various forms of 
writing and various writing techniques. 

Mathematics Computation. Test 3 contains 48 items that measure understanding of the 
operations of addition, subtraction, multiplication, and division. Depending on the level of 
the test, content includes whole numbers, decimals, fractions, algebraic expressions, 
percents, and exponents. 



E (Easy) 



2.6 
4.6 
6.6 
8.6 



4.9 
6.9 
8.9 
12.9 



M (Medium) 
D (Difficult) 
A (Advanced) 
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Mathematics Concepts and Applications. Test 4 contains 40 items that measure 
understanding of mathematics concepts. Specific skills include numerations, number 
sentences, number theory, problem solving, measurement, and geometry. 

Language Mechanics. Test 5 contains 30 items that measure skills in the mechanics of 
capitalization and punctuation. Editing skills are measured in the context of passages 
presented in various formats. 

Language Expression. Test 6 contains 45 items that measure skills in language usage and 
sentence structure. The items measure skills in the use of various parts of speech, formation 
and organization of sentences and paragraphs, and writing for clarity. All items on the test 
are based on rules of written standard English. 

Spelling. Test 7 contains 30 items that measure applications of spelling rules for consonants, 
vowels, and various structural forms. Items are presented in the context of sentences with a 
missing word. The examinee identifies the correct spelling of the word that would complete 
the sentence. 

The TABE has a Locator Test that can be used to determine the appropriate level of the 
TABE to administer to each examinee. The approximate total time for the administration of 
the TABE is four and one-half hours. 

Validity and Reliability 

The validity of a test indicates the degree to which the test measures what it purports to 
measure. The aim of the TABE is to provide valid measurement of basic skills in reading, 
mathematics, language and spelling. Extensive validity measures were conducted during the 
development of the TABE. Details can be found in the Test of Adult Basic Education, Forms 
5 and 6, Technical Report. Most adult educators consider the TABE to be a valid test. 

The reliability of a test is a measure of the consistency of test results. A reliable test 
produces scores that remain relatively stable when the test is administered repeatedly under 
similar conditions. Extensive reliability studies have been conducted on the TABE. Details 
can be found in the Tests of Adult Basic Education, Forms 5 and 6, Technical Report. The 
TABE is considered to be a very reliable test. 

Advantages and Disadvantages 

Standardised tests such as the TABE are commonly used by community colleges across the 
country and provide a good assessment of an individual's general basic skill abilities. • In 
addition, standardized tests can be used as program pre- and post-assessments to measure 
gains in general reading abilities. The results are depicted in grade levels, which are easily 
understood by program participants as well as those who are funding or sponsoring the 
program. 
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There are several disadvantages to using only standardized tests in a workplace literacy 
program. Standardized tests measure general basic skill abilities and not the specialized 
literacy skills required in the workplace. A learner could move from the fifth grade reading 
level to the eighth grade reading level and therefore be better able to read newspapers and 
magazines. However, the learner is not likely to be able to transfer these skills easily to 
reading a SPC chart, a technical training manual, or a job safety analysis. Unfortunately, 
gains made in teaching individuals the types of skills needed to improve reading skills for 
work-related materials may be only partially reflected in standardized tests. 
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TESTS OF APPLIED LITERACY SKILLS 



The Tests of Applied Literacy Skills (TALS) are the most recent and sophisticated in a long 
series of efforts to develop measures of an individual's facility in using printed and written 
materials. The TALS are designed to assist program providers and staff in estimating the 
♦ literacy proficiencies of individuals applying for or receiving services; to assist in placing 
individuals into appropriate, existing programs; and to provide a means for assessing learner 
progress over time. 

Literacy Scales 

The TALS have three scales, prose literacy, document literacy and quantitative literacy: 

Prose literacy - the knowledge and skills needed to understand and use information from 
texts that include editorials, news stories, poems, and fiction; for example, finding a piece of 
information in a newspaper article, interpreting instructions from a warranty, inferring a 
theme from a poem, or contrasting views expressed in an editorial. 

Document literacy - the knowledge and skills required to locate and use information 
contained in materials that include job applications, payroll forms, transportation schedules, 
maps, tables, and graphs; for example, locating a particular intersection on a street map, 
using a schedule to choose the appropriate bus, or entering information on an application 
form. 

Quantitative literacy - the knowledge and skills required to apply arithmetic operations, 
either alone or sequentially, using numbers embedded in printed materials; for example, 
balancing a checkbook, figuring out a tip, completing an order form, or determining the 
amount of interest from a loan advertisement. 

Individuals assessed by the TALS are not rated as either "literate" or "illiterate," rather they 
are classified as falling into one of five literacy levels for each of the three scales. The five 
levels on each scale reflect shifts in the literacy skills and strategies required to perform 
increasingly complex tasks. Tasks at the lower end of each scale differ significantly from 
those at the high end. In fact, the tasks along each scale follow a clear progression in 
information-processing skills and strategies. 

On the prose scale, for example, tasks with low scale values ask readers to locate or identify 
information in brief, familiar, or uncomplicated materials, while those at the high end ask 
them to perform more demanding activities using materials that tend to be lengthy, 
unfamiliar, or complex. Similarly, on the document and quantitative scales, the tasks at the 
low end of the scale differ from those at the high end in terms of the structure of the 
material, the content and context of the material, and the nature of the directive. 
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SAMPLE TASKS THAT CAN BE PERFORMED BY INDIVIDUALS AT EACH 
LITERACY LEVEL FOR EACH LITERACY SCALE 




Prose 


Document 


Quantitative 


Level 1 


Identify a piece of 
specific information in 
a brief news article. 


Locate the time or place 
of a meeting on a form. 


Total an entry on a 
deposit slip. 


Level 2 


Locate information in 
text, make low-level 
inferences using 
printed materials, and 
integrate easily 
identifiable pieces of 
information. 


Locate a particular 
intersection on a street 
map, and enter 
background information 
on a simple form. 


Perform quantitative 
tasks that involve a 
single operation where 
the numbers are either 
stated or can be easily 
found in text, e.g. 
calculate the total cost 
of a purchase or 
determine the difference 
in price between two 
items. 


Level 3 


Integrate information 
from relatively long or 
dense text. 


Integrate information 
from documents. 


Determine the 
appropriate arithmetic 
operation based on 
information contained in 
the directive, and 
identify the quantities 
needed to perform that 
operation. 


Level 4 


Explain the difference 
between two types of 
employee benefits, and 
compare two 
metaphors used in a 
poem. 


Use a bus schedule to 
determine the 
appropriate bus for a 
given set of conditions, 
and use a table of 
information to determine 
the pattern in oil exports 
across a number of 
years. 


Determine the correct 
change using 
information in a menu, 
and using information 
stated in a news article, 
calculate the amount of 
money that should go to 
raising a child. 
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SAMPLE TASKS THAT CAN BE PERFORMED BY INDIVIDUALS AT EACH 
LITERACY LEVEL FOR EACH LITERACY SCALE (continued) 




Prose 


Document 


Quantitative 


Level 5 


Summarize two ways 
lawyers may challenge 
prospective jurors, and 
interpret a brief phrase 
from a lengthy news 
article. 


Use information in a 
table to complete a 
graph, including labeling 
the axes, and use a table 
depicting information 
about parental 
involvement in a school 
survey to write a 
paragraph summarizing 
the extent to which 
parents and teachers 
agree. 


Determine the shipping 
and total costs on an 
order form for items in 
a catalog, and using a 
calculator, determine 
the total cost of carpet 
to cover a room. 



While the literacy levels on each scale can be used to explore a range of literacy demands, 
these data do not reveal the types of literacy demands that are associated with particular 
contexts in this pluralistic society. That is, they do not enable us to say what specific level 
of prose, document, or quantitative skill is required to obtain, hold, or advance in a 
particular occupation, to manage a household, or to obtain legal or community services, for 
example. Nevertheless, to the extent that the tasks at a given level correspond to those in 
your workplace, using these scales enables you to assess the needs of your workers in your 
literacy program, and then prescribe the appropriate curriculum. 

Test Administration 

Adults have up to 40 minutes to complete each of the TALS, and each test takes 
approximately five minutes to correct and score. The tests themselves: 



Use open-ended simulation tasks; 

Emphasize tasks that measure a broad range of information-processing skills and 
cover a wide variety of contexts; 

Emphasize simulation tasks that require brief written and/or oral responses; 

Emphasize tasks that ask respondents to describe how they would set up and solve a 
problem; and 

Use a simple, four-function calculator to solve selected quantitative problems. 



114 



ERLC 



-A- A* Jt 



Because the TALS were used for the 1992 National Adult Literacy Survey (the results of 
which were released in September, 1993), scores on the TALS can be linked to large-scale 
assessments of adult literacy. 

[NOTE: Text for this section was taken from Adult Literacy in America: A First Look at the 
Results of the National Adult Literacy Survey, by the Educational Testing Service under 
contract with the National Center for Education Statistics, Office of Educational Research 
and Improvement, U.S. Department of Education, September, 1993; ETS Tests of Applied 
Literacy Skills: Administration and Scoring Manual, by the Educational Testing Service, 
1991; and the ETS Tests of Applied Literacy Skills marketing brochure.] 
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CERT AINTEED' S EMPLOYEE LEARNING PREFERENCE SURVEY 



Roofing products group plants are undergoing changes to improve their long-term 
competitiveness and to ensure they continue to produce the highest quality roofing products. 
As a part of implementing these changes, CertainTeed is also instituting an employee 
development program called Skill and Knowledge in Lifelong Learning or "SKILL" training. 
This questionnaire is designed to help us plan training courses and programs that meet your 
learning style preferences and address your training interests as well as meet the company's 
needs. 

Please help us by completing this survey and returning it to the person designated to receive 
the surveys in your plant. Your response is absolutely confidential. Do not write your name 
on this form. Your response will be tabulated with all others from your work area in all 
three roofing products group plants. 



PLANT: In which plant do WORK AREA: In what area of 

you work? the plant do you work? 



□ 


Oxford, NC 


□ 


Warehouse 


□ 


Avery, OH 


□ 


Stillyard 


□ 


Shakopee, MN 


□ 


Felt Mill 






□ 


Maintenance 






□ 


Production Line 






□ 


Office 



DEMOGRAPHIC DATA. This information is being collected only for statistical analysis. 


All responses will be grouped together into statistical summaries. Individual survey forms 


will be discarded. 






AGE: Please indicate the age 


EDUCATIONAL LEVEL: What is the 


grouping within which you fall. 


highest level of formal schooling that you have completed? 


□ 17 to 24 


□ 


less than 6th grade 


□ 25 to 34 


□ 


6th to 8th grade 


□ 35 to 44 


□ 


some high school 


□ 45 to 54 


□ 


high school diploma/GED 


□ over 55 


□ 


some classes after high school 




□ 


college degree 
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PLEASE RESPOND TO EACH ITEM 

WHAT IS THE TITLE OF YOUR CURRENT JOB? 

HOW LONG HAVE YOU BEEN IN YOUR CURRENT JOB?_ 
HOW WERE YOU TRAINED FOR YOUR CURRENT JOB?__ 

Check as many as apply: 

□ / received no training, 

□ / was trained by the person who held the job before me, 

□ / received training at the plant, 

□ / attended a technical school or a community college, 

□ / took courses from a correspondence school, 

□ / received training from CertainTeed (Corporation or group), 

□ Other (please describe) 
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PRIOR SCHOOLING: HAVE YOU COMPLETED ANY FORMAL OR 
INFORMAL CLASSES OR TRAINING IN ANY OF THE FOLLOWING 
AREAS? 

Include training that you have taken outside of school as well as courses taken at a 
school or college. 

Check boxes where you have received some training: 



A. Mathematics 

- General Math . □ 

- Business Math □ 

- Algebra □ 

- Geometry □ 

- Trigonometry □ 

- Statistics □ 

- Shop Math □ 

B. Science 

- General Science □ 

- Chemistry □ 

- Engineering □ 

- Physics □ 
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C. Vocational/Industrial 



- Drafting/Blueprint Reading 



□ 



- Metalworking 



□ 



- Electricity 



□ 



- Electronics 



□ 



- Power (hydraulics/pneumatics) 



□ 



- Computers 



□ 



- Technical Writing 



□ 



SECOND THOUGHTS: WHICH OF THE COURSES LISTED BELOW 
WOULD YOU LIKE TO TAKE OR REVIEW AGAIN, IF YOU HAD A 
SECOND CHANCE? 



Check all that apply: 



□ 


Basic 


□ 


Spelling/Vocabulary 


□ 


Basic Writing 


□ 


Technical Writing 


□ 


Learning Skills 


□ 


Study Skills 


□ 


Basic Arithmetic 


□ 


General Math 


□ 


Shop Math 


□ 


Algebra 


□ 


Geometry 


□ 


Trigonometry 


□ 


Statistics 


□ 


General Science 


□ 


Chemistry 


□ 


Engineering 


□ 


Physics 


□ 


Drafting/Blueprint Reading 


□ 


Metalworking 


□ 


Electronics 


□ 


Power (hydraulics/pneumatics) 


□ 


Computer Programming 
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□ Computer Integrated Manufacturing 

□ Programmable Logic Controllers 



COMMUNICATIONS: PLEASE RATE YOURSELF IN EACH OF THE 
AREAS BELOW BY PLACING A CHECKMARK IN THE COLUMN THAT 
BEST DESCRIBES YOU. 



Need Help OK Very Well 



A. Speaking: How well can you 



Ask for assistance/directions? □ □ □ 

Help-out a co-worker? □ □ □ 

Instruct a group of 3-5 people? □ □ □ 

Give a report to a larger group? □ □ □ 

Conduct a meeting? □ □ □ 



B. Reading: Can you read and understand - 



Technical manuals and reports 
Job Safety Analysis (JSA) sheets 
Job procedures and checklists 
Process flowcharts 
Blueprints and schematics 
Newspapers 

C. Writing: How well can you 

Make understandable entries 
into a logbook 

Outline or organize your ideas 
Spell and punctuate 
Write up a suggestion or 
process improvement 

D. Mathematics: How well can you — 

Add, subtract, multiply* and divide 
Use fractions and decimals 
Chart values on a control graph 
Calculate mean and standard deviation 
for a normal distribution 



□ □ □ 

□ □ □ 

□ □ □ 

□ □ □ 

□ - □ □ 

□ □ □ 



□ □ □ 

□ □ □ 

□ □ □ 

□ □ □ 



□ □ □ 

□ □ □ 

□ □ □ 

□ □ □ 
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Need Help 



OK Very Well 



E. Measuring: How well do you use/understand - 



A tape measure □ □ □ 

Vernier calipers □ □ □ 

A micrometer □ □ □ 

Temperature gauges □ □ □ 

Go/no-go gauges □ □ □ 

Volt/Ohm meters □ □ □ 

Pressure gauges □ □ □ 



WHAT ARE THE BEST WAYS FOR YOU TO LEARN SOMETHING? 

Listed below are some ways a person can learn. Please read each one, and tell us if 
you like to learn that way. Put a check under "a lot" if you really like to learn new 
things that way. Put a check under "some" if it would be OK to learn that way. Put 
a check under "not at all" if you don't like to learn that way. 





A Lot 


oome 


XT - f — 4- A 11 

JNot at All 


- working with my hands 


□ 


□ 


□ 


- learning in a group 


□ 


□ 


□ 


- learning on my own by reading 








or practicing 


□ 


□ 


□ 


- learning from TV programs 


□ 


□ 


□ 


- seeing something for myself 








rather than being told 


□ 


□ 


□ 


- using a computer to learn 


□ 


□ 


□ 


- seeing films or videotapes 


□ 


□ 


□ 


- listening to a speech 


□ 


□ 


□ 


- listening to a lecture and 








taking notes 


□ 


□ 


□ 


- doing worksheets and practice problems 


□ 


□ 


□ 


- having someone give examples 


□ 


□ 


□ 


- having someone show me how 








to do something 


□ 


□ 


□ 


- practicing something by myself 








until I get it 


□ 


□ 


□ 


- working with another person 


□ 


□ 


□ 


- asking questions 


□ 


□ 


□ 


- learning under pressure when 








there is a deadline 


□ 


□ 


□ 


- memorizing 


□ 


□ 


□ 
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WHEN DO YOU LIKE TO STUDY? 



Check any that apply to you: 

□ when it 's quiet 

□ when other people are around but they are quiet 

□ when other people are around and they are talking 

□ when you are by yourself 

□ other (please describe): 

DO YOU WANT YOUR WORK CHECKED BY AN INSTRUCTOR RIGHT 
AFTER YOU COMPLETE IT? 

Check one: 

□ yes 

□ it doesn 7 matter 

WHEN DO YOU BEST UNDERSTAND DIRECTIONS? 

Check one: 

□ when you are told what they are 

□ when you read them for yourself 

□ it doesn 9 t matter 

□ it depends on 
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11 . HOW OFTEN DO YOU READ FOR PLEASURE (BOOKS, ARTICLES, 
MAGAZINES)? 

Check one: 

□ every day 

□ two or three times a week 

□ once a week 

□ hardly ever 

□ never 

12. HOW OFTEN DO YOU READ ON THE JOB? 

Check one: 

□ every day 

□ two or three times a week 

□ once a week 

□ hardly ever 

□ never 

13. WHEN YOU DO NOT UNDERSTAND SOMETHING, IS IT.... 

Check one: 

□ easy for you to ask questions? 

□ hard for you to ask questions? 
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EVALUATION APPROACHES 



Correlational Research 

Correlational research approaches allow for the investigation of the extent to which variations 
in one factor correspond with variations in one or more other factors. An example of this 
approach in this study was an analysis of the relationships between the number of shipping 
errors committed by an individual and the number of hours spent in basic skills training. 
Ideally, we were searching for a direct negative correlation between the variables — that is, 
as training hours increased, shipping errors would decrease. 

Causal-Comparative Research 

Causal-comparative research approaches allow for the investigation of possible cause and 
effect relationships by observing some existing consequence and searching back through the 
data for plausible causal factors. The limitation of causal-comparative approaches is that one 
cannot infer cause and effect. In most cases, all that can be concluded is that a relationship 
between variables exists. An example of this approach in- this study was the exploration of 
the differences in characteristics in terms of selected dependent variables between crews 
grouped according to measures of the quality of the products they produced. CertainTeed's 
hope was to establish a positive relationship between individuals who were members of high 
performing crews and such variables as Test of Adult Basic Education gain scores and level 
of participation in the Precision Strike program. 

Quasi-Experimental Research 

Quasi-Experimental research approaches allow for the approximation of true experimental 
conditions in a setting that does not allow the control and/or manipulation of all relevant 
variables. The researcher must clearly understand what compromises exist in the internal 
and external validity of the design and proceed within these limitations. An example of this 
approach in this study was the exploration of the relationship between Test of Adult Basic 
Education (TABE) gain scores and participation in the Precision Strike program. 
Participation in the program was voluntary, and some of the workforce at CertainTeed did 
not participate at all. However, we were fortunate in that pre- and post-TABE tests were 
administered to almost all of the, workforce, allowing for the establishment of a non- 
equivalent control group. We were able to compare the TABE gain scores of individuals 
who participated in the program with those who did not. 
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CERTAINTEED'S EMPLOYEE SELF-ASSESSMENT SURVEY 



Job Title 



Date 



Crew 



Name or Precision Strike Program 
I.D. Number (Optional) 



Precision Strike Program Participant Yes. 



No. 



Your participation in this survey is critical to the Precision Strike program evaluation. 

Please check one answer to each question. Try very hard to be open and honest when 
answering the question. Remember, this data will be kept confidential. The only person that 
will see individual data is Fred Frederick, the Precision Strike program evaluator. A 
summary of the results of this survey will be presented to CertainTeed management. 

1. 1 understand the plant goals and work to meet them. 

never seldom sometimes most of the time always 

2. I find ways to solve problems. 

never seldom sometimes most of the time always 

3. I attend to all the important details when doing a job. 

never seldom sometimes most of the time always 

4. I get things done well and on time. 

never seldom sometimes most of the time always 

5. I often speak out and add to discussions. 

never seldom sometimes most of the time always 

6. I contribute ideas which are practical as well as technically sound, 
never seldom sometimes most of the time always 
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7. I am very much concerned about the quality of my work. 

never seldom sometimes most of the time always 

8. I attend to the job at hand and keep at it. 

never seldom sometimes most of the time always 

9. I know how to get things done or figure out how to do them, 
never seldom sometimes most of the time always 

10. I try hard to get things right the first time. 

never seldom sometimes most of the time always 

11. I can be counted on to hold up my end of the job. 

never seldom sometimes most of the time always 

12. I work independently. 

never seldom sometimes most of the time always 

13. I work well without close supervision. 

never seldom sometimes most of the time always 

14. I can read and understand job-related reading material. 

never seldom sometimes most of the time always 

15. I know the technical parts of my work well. 

never seldom sometimes most of the time always 

16. I am helpful and lend a hand to help other people in my crew and other crews, 
never seldom sometimes most of the time always 

17. I am open to blunt and direct feedback. 

never seldom sometimes most of the time always 
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18. I profit from blunt and direct feedback. 

never seldom sometimes most of the time always 

19. I readily accept suggestions from other people. 

never seldom sometimes most of the time always 

20. I put suggestions to good use. 

never seldom sometimes most of the time always 

21. I handle problems easily. 

never seldom sometimes most of the time always 

22. I do not hesitate to say so if I think something is wrong. 

never seldom sometimes most of the time always 

23. I seldom wander off the track. 

never seldom sometimes most of the time always 

24. I can do all of the math required for my job. 

never seldom sometimes most of the time always 

25. I know how to calculate the average (mean) of a set of numbers, 
never seldom sometimes most of the time always 

26. I know how to calculate the range of a set of numbers. 

never seldom sometimes most of the time always 

27. I can use a tape measure to accurately measure an object to within l/16th of an inch, 
never seldom sometimes most of the time always 

28. I can read and understand all Job Safety Analyses that apply to my job. 
never seldom sometimes most of the time always 



127 



29. I am an eager and motivated worker, 

never seldom sometimes most of the time always 

30. I volunteer for assignments that are harder and more interesting, 
never seldom sometimes most of the time always 

31. I create ways to check my own work and other's work. 

never seldom sometimes most of the time always 

32. I change workplace materials or procedures on my job that are confusing or don't 
work well. 

never seldom sometimes most of the time always 

33. I volunteer to write reports. 

never seldom sometimes most of the time always 

34. I volunteer to speak or give reports at group meetings. 

never seldom sometimes most of the time always 

35. I identify problems in the quality of the product. 

never seldom sometimes most of the time always 

36. I actively seek solutions to problems with the quality of the product, 
never seldom sometimes most of the time always 

37. I identify problems in the production process. 

never seldom sometimes most of the time always 

38. I actively seek solutions to problems in the production process, 
never seldom sometimes most of the time always 

39. I understand the terms used in the roofing business. 

never seldom sometimes most of the time always 
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40. I give my ideas at meetings. 

never ; seldom sometimes most of the time always 

41. I write in the log book. 

never seldom sometimes most of the time always 

42. I attend my process improvement team meetings. 

never seldom sometimes most of the time: always 

43. I see the importance of learning new things. 

never seldom sometimes most of the time always 

44. I feel comfortable learning new things. 

never seldom sometimes most of the time always 

45. I break up difficult or complicated tasks or problems into smaller parts that are easier 
to handle. 

never seldom sometimes most of the time always 

46. I ask questions to get information to help me understand a task or problem, 
never seldom sometimes most of the time always 

47. I catch and correct my own errors. 

never seldom sometimes most of the time always 

48. I schedule time to meet deadlines. 

never seldom sometimes most of the time always 

49. I am able to use new information or knowledge in my job. 

never seldom sometimes most of the time always 

50. I listen closely when my supervisor explains something to me. 
never seldom sometimes most of the time always 
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51. I understand the meaning of new information given to me as well as the explanation, 
never seldom sometimes most of the time always 

52. I keep asking "how" and "why" questions until I understand. 

never seldom sometimes most of the time always 

53. I like to enter into discussions in training programs or in other group meetings such as 
process improvement team meetings. 

never seldom sometimes most of the time always 

54. I can explain tasks and methods to others in my crew. 

never seldom sometimes most of the time always 

55. I understand the pros and cons of a suggestion or idea. 

never seldom sometimes most of the time always 

56. I express questions in a clear manner. 

never seldom sometimes most of the time always 

57. I organize time well. 

never seldom sometimes most of the time always 

58. I ask relevant questions that show my active involvement in discussions, 
never seldom sometimes most of the time always 

59. I connect new ideas to old ideas in trying to figure out tasks and problems, 
never seldom sometimes most of the time always 

60. If you are a Precision Strike program participant, what can you do now that you 
couldn't do before you started participating in the program? 
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61. If you could change anything about the Precision Strike program, what changes would 
you make? 



62. If you are participating in the Precision Strike program, briefly describe why you are 
participating and what you expect to gain by your participation. 



63. If you are not participating in the Precision Strike program, briefly describe why you 
are not participating. 



THANK YOU VERY MUCH FOR COMPLETING THIS SURVEY AND HELPING 
US TO EVALUATE THE PRECISION STRIKE PROGRAM. PUT THE 
COMPLETED SURVEY INTO THE ENVELOPE PROVIDED AND RETURN IT 
TO FRED FREDERICK. 
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CERTAINTEED'S SUPERVISOR RATING SURVEY 



Job Title 



Date 



Crew 



Name or Precision Strike Program 
LD. Number (Optional) 



Precision Strike Program Participant Yes. 



No 



Your participation in this survey is critical to the Precision Strike program evaluation. 

Please check one answer to each question. Try very hard to be open and honest when 
answering the question. Remember, this data will be kept confidential. The only person that 
will see individual data is Fred Frederick, the Precision Strike program evaluator. A 
summary of the results of this survey will be presented to CertainTeed management. 

This employee: 

1. Understands the plant goals and works to meet them. 

never seldom sometimes most of the time always 

2. Finds ways to solve problems. 

never seldom sometimes most of the time always 

3. Attends to all the important details when doing a job. 

never seldom sometimes most of the time always 

4. Gets things done well and on time. 

never seldom sometimes most of the time always 

5. Often speaks out and adds to discussions. 

never seldom sometimes most of the time always 

6. Contributes ideas which are practical as well as technically sound, 
never seldom sometimes most of the time always 
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Is very much concerned about the quality of his/her work. 

never seldom sometimes most of the time always — 

Attends to the job at hand and keeps at it. 

never seldom sometimes most of the time always — 

Knows how to get things done or figures out how to do them. 

never seldom sometimes most of the time always — 

Tries hard to get things right the first time. 

never seldom sometimes most of the time always — 

Can be counted on to hold up his/her end of the job. 

never seldom sometimes most of the time always — 

Works independently. 

never seldom sometimes most of the time always 

Works well without close supervision. 

never seldom sometimes most of the time always — 

Can read and understand job-related reading material. 

never seldom sometimes most of the time always — 

Knows the technical parts of his/her work well. 

never seldom sometimes most of the time always — 

Is helpful and lends a hand to help other people in his/her crew and other crews. 

never seldom sometimes most of the time always — 

Is open to blunt and direct feedback. 

never seldom sometimes " most of the time always — 
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18. Profits from blunt and direct feedback. 

never seldom sometimes most of the time always 

19. Readily accepts suggestions from other people. 

never seldom sometimes most of the time always 

20. Puts suggestions to good use. 

never seldom sometimes most of the time always 

21. Handles problems easily. 

never seldom sometimes most of the time always 

22. Does not hesitate to say so if he/she thinks something is wrong, 
never seldom sometimes most of the time always 

23. Seldom wanders off the track. 

never seldom sometimes most of the time always 

24. Can do all of the math required for his/her job. 

never seldom sometimes most of the time always 

25. Knows how to calculate the average (mean) of a set of numbers, 
never seldom sometimes most of the time always 

26. Knows how to calculate the range of a set of numbers. 

never seldom sometimes most of the time always 

27. Can use a tape measure to accurately measure an object to within l/16th of an inch, 
never seldom sometimes most of the time always 

28. Can read and understand all Job Safety Analyses that apply to his/her job. 
never seldom sometimes most of the time always 
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29. Is an eager and motivated worker. 

never seldom sometimes most of the time always 

30. Volunteers for assignments that are harder and more interesting, 
never seldom sometimes most of the time always 

31. Creates ways to check his/her work and other's work. 

never seldom sometimes most of the time always 

32. Changes workplace materials or procedures on his/her job that are confusing or don't 
work well. 

never seldom sometimes most of the time always 

33. Volunteers to write reports. 

never seldom sometimes most of the time always 

34. Volunteers to speak or give reports at group meetings. 

never seldom sometimes most of the time always 

35. Identifies problems in the quality of the product. 

never seldom sometimes most of the time always 

36. Actively seeks solutions to problems with the quality of the product, 
never seldom sometimes most of the time always 

37. Identifies problems in the production process. 

never seldom sometimes most of the time always 

38. Actively seeks solutions to problems in the production process, 
never seldom sometimes most of the time always 

39. Understands the terms used in the roofing business. 

never seldom sometimes most of the time always 

135 



t A - 



ERIC 



40. Gives his/her ideas at meetings. 

never seldom sometimes most of the time always 

41. Writes in the log book. 

never seldom sometimes most of the time always 

42. Attends process improvement team meetings. 

never seldom sometimes most of the time always 

43. Sees the importance of learning new things. 

never seldom sometimes most of the time always 

44. Feels comfortable learning new things. 

never seldom sometimes most of the time always 

45. Breaks up difficult or complicated tasks or problems into smaller parts that are easier 
to handle. 

never_ seldom sometimes most of the time always 

46. Asks questions to get information to help understand a task or problems, 
never seldom sometimes most of the time always 

47. Catches and corrects his/her own errors. 

never seldom sometimes most of the time always 

48. Schedules time to meet deadlines. 

never seldom sometimes most of the time always 

49. Is able to use new information or knowledge in his/her job. 

never seldom sometimes most of the time always 

50. Listens closely when his/her supervisor explains something. 

never seldom sometimes most of the time always 
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51. Understands the meaning of new information given to him/her as well as the 
explanation. 

never seldom sometimes most of the time always 

52. Keeps asking "how" and "why" questions until he/she understands, 
never seldom sometimes most of the time always 

53. Likes to enter into discussions in training programs or in other group meetings such 
as process improvement team meetings. 

never seldom sometimes most of the time always 

54. Can explain tasks and methods to others in his/her crew. 

never seldom sometimes most of the time always 

55. Understands the pros and cons of a suggestion or idea. 

never seldom sometimes most of the time always 

56. Expresses questions in a clear manner. 

never seldom sometimes most of the time always 

57. Organizes time well. 

never seldom sometimes most of the time always 

58. Asks relevant questions that show active involvement in discussions, 
never seldom sometimes most of the time always 

59. Connects new ideas to old ideas in trying to figure out tasks and problems, 
never seldom sometimes most of the time always 

60. If you could change anything about the Precision Strike program, what changes would 
you make? 
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If you are participating in the Precision Strike program, briefly describe why you are 
participating and what you expect to gain by your participation. 



If you are not participating in the Precision Strike program, briefly describe why you 
are not participating. 



THANK YOU VERY MUCH FOR COMPLETING THIS SURVEY AND HELPING 
US TO EVALUATE THE PRECISION STRIKE PROGRAM. PUT THE 
COMPLETED SURVEY INTO THE ENVELOPE PROVIDED AND RETURN IT 
TO FRED FREDERICK. 
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CERTAINTEED'S STRUCTURED INTERVIEW GUIDES 
Guide 1 

1 . What are your personal goals in participating in the Precision Strike program? 

2. What are your professional goals in participating in the Precision Strike program? 



3. You have been through an orientation to the center. What are your first impressions 
(facility, instructor, hours, curriculum, delivery methods)? 



4. How do you think you will work time for going to the center into your present 
schedule (what might make it difficult)? 



5. What do you hear people saying about the company's policies concerning the 
Precision Strike program? 



6. Other comments? 
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Guide 2 

1 . Have you been attending Precision Strike training? 
How often? 

What are you studying? 
How helpful are the instructors? 
What do you like about the program? 
What do you dislike about the program? 

2. What do you think we could do to get more people coming to the center? 

3. Do you think attendance should be mandatory? 

4. What are the biggest drawbacks to attending the center? 

5. Anything else you thing could help improve the program? 
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Guide 3 

1. Are you presently participating in Precision Strike? (If not, why not?) 
(If you are not participating, do you plan to do so? When?) 

In what areas? 

Approximately how many hours per week/month? 

Do you visit the center on a regular schedule? 

What factors would encourage you to participate more? 

2. Are your educational goals being met through Precision Strike? 

Can you see any difference in your work performance because of Precision Strike? 

How has Precision Strike changed your work performance? 

In what other ways has Precision Strike affected your life? 

How could Precision Strike be improved to have a greater impact on your job? 

How could Precision Strike be changed to be more helpful to you outside the job? 

3. Is there any way the Precision Strike staff could improve to better serve your needs' 

4. Other comments? 
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APPENDIX II: CERTAINTEED EXAMPLES 



CERTAINTEED SKILL PROFILES FOR THE BASIC SKILLS AREA BY TRAINING 
LEVEL 

Training Level I 
Learning Skills 

1. Recognize and distinguish between different learning styles, and identify personal, 
preferred learning style. 

2. Organize learning activities. 
Reading Skills 

1. Distinguish between different reading purposes (reading to learn, reading to do, 
reading to assess). 

2. Recognize common words and meanings, task- related words with technical meanings, 
and meanings of common abbreviations and acronyms. 

3. Follow sequential directions and/or illustrations to complete a task. 

4. Locate pages, titles, paragraphs, figures, or charts needed to answer questions or 
solve problems. 

5. Use a completed form to locate information to complete a task. 

6. Use a table of contents, index, appendix, glossary, system, or subsystem. 

7. Determine the presence of a defect or extent o.' damage. 

o. Classify or match objects by color, size, or significant marking. 

9. Use common knowledge for safety. 

10. Apply preventive measures prior to task to minimize problems. 

11. Select the appropriate course of action in an emergency. 

12. Identify details, labels, and numbers. 
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13. Follow sequenced illustrations as a guide. 
Writing Skills 

1. Enter single job-related words to a form. 

2. Legibly record short spoken statements. 
Computation Skills 

1. Read, write, and count single and multiple digit whole numbers. 

2. , Add, subtract, multiply, and divide single and multiple digit numbers. 

3. Read and write common fractions. 

4. Read and write decimals in one or more places. 

5. Read and write percentages. 

6. Read numbers or symbols from a weight measuring scale. 

7. Use a measuring rule to determine an object's physical dimensions in standard units. 

8. Read and tell time. 
Training Level II 
Learning Skills 

All those at Training Level I plus: 

3. Use several different strategies to relate, process, and recall new information. 

4. Apply the appropriate type of thinking (convergent, divergent, critical, or intuitive) to 
each learning activity. 

Reading Skills 

All those at Training Level I plus: 

14. Recognize job-specific words with technical meanings and job-specific abbreviations 
and acronyms. 
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15. Locate individual specifications or facts in text. 

16. Combine information from multiple sources that contribute to the completion of a 
task. 

17. Select parts of text or visual materials to complete a task. 

18. Identify similarities and differences in objects. 

19. Apply information to locate malfunctions or decide upon a course of action. 

20. Interpret codes and symbols. 
Writing Skills 

All those at Training Level I plus: 

3. Enter Written information (words, phrases) on forms. 

4. Spell task-related and job-related technical words, abbreviations, and acronyms 
correctly. 

5. Transfer numbers, codes, dates, and figures from equipment or machinery into 
appropriate sections of forms, 

6. Generate written communications according to a specific format. 
Computation Skills 

All those at Training Level I plus: 

9. Use addition, subtraction, multiplication, and division to solve problems with single 
and multiple digit numbers. 

10. Add, subtract, multiply, and divide common fractions. 

11. Add, subtract, multiply, and divide decimals in one or more places. 

12. Read, write, and compute using percentages. 

13. Relate fractions, decimals, and percentages. 

14. Read numbers or symbols from machine counting devices. 
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15. Use a measuring device to determine an object's weight in standard units. 
Training Level III 

Learning Skills 

All those at Training Levels I and II plus: 

5. Use lists and notes to store and retrieve information. 

Reading Skills 

All those at Training Levels I and II plus: 

21. Recognize computer related jargon and acronyms. 

22. Identify factual details and specifications within text to complete a task. 

23. Read text to determine the main idea of a paragraph or section. 

24. Read and interpret flow charts and schematics which sequence events or structure 
processes. 

25. Organize information from multiple sources into a series. 
Writing Skills 

All those at Training Levels I and II plus: 

7. Organize information into paragraphs and sentences. 

8. Record essential information accurately and precisely in log books using phrases or 
simple sentences. 

9. Outline a process by arranging key events sequentially into a step-by-step process. 
Computation Skills 

All those at Training Levels I and II plus: 

16. Round off single and multiple digit numbers, fractions, and decimals. 

17. Solve occupational word problems with common fractions and decimals. 
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18. Recognize such concepts as parallel, perpendicular, triangle, rectangle, square, and 
circle. 

19. Use a calculator to perform basic arithmetic operations to solve problems. 

20. Understand and apply principles of central tendency and statistics: mean, mode, 
median, range, and dispersion. 

21. Use coordinate systems (tables, charts, graphs, maps). 

22. Read and interpret complex numbers and symbols from gauges and display screens. 
Training Level IV 

Learning Skills 

All those at Training Levels I, II, and III plus: 

6. Apply new knowledge to existing job duties and tasks. 

7. Adapt easily to changing company needs through retraining and new learning. 
Reading Skills 

All those at Training Levels I, II, and III plus: 

26. Cross reference within and across source materials to select information to perform a 
routine. 

27. Distinguish between relevant and irrelevant information in text or visuals. 

28. Use skimming or scanning to determine if text contains relevant information. 

29. Read two or more column charts to obtain information. 

30. Locate chart information at the intersection of rows and columns. 

31. Apply information from tables or graphs to locate a malfunction or select actions. 

32. Use flow charts to sequence events or solve problems. 

33. Isolate problems in schematics, tracing the cause of the problems. 

34. Read and interpret blueprints and blueprint symbols. 
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Writing Skills 

All those at Training Levels I, II, and III plus: 

10. Summarize events in an accurate and complete manner. 

11. Limit subject and focus of writing according to purpose, audience, and format. 

12. Write a technical report that includes supporting documentation, justifications for 
actions taken, and reasons for rejecting alternative actions. 

13. Use outlines and diagrams to plan, remember, and organize information and actions. 
Computation Skills 

All those at Training Levels I, II, and III plus: 

23. Solve math problems by selecting and using the correct order of operations. 

24. Solve occupational word problems using more than one step and several computational 
operations. 

25. Perform calculations quickly. 

26. Compute mean, mode, median, standard deviation, dispersion, and range for a normal 
distribution of processing variables. 

27. Read and construct x-bar and R-bar charts, and scatter plots that correlate x and y 
values. 
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CERTAINTEED SKILL PROFILES FOR THE FUNCTIONAL SKILLS AREA BY 
TRAINING LEVEL 

Training Level I 

Communication Skills 

1. Listen for content. 

2. Follow spoken directions. 

3. Extract information from extended messages. 

4. State information concisely and efficiently. 
Group Effectiveness 

1. Contribute information in keeping with the topic. 

2. Express opinions and judgments (use feelings or intuition). 
Problem Solving and Analytical Thinking 

1. Recognize task specific problems and take appropriate action. 

Training Level II 

Communication Skills 

All those at Training Level I plus: 

5. Understand how listening style affects both the transmission and reception of spoken 
information. 

6. Follow detailed verbal directions to complete a task. 

7. Recognize and interpret non-verbal information (e.g., tone, gesture, voice inflection, 
body language, attentiveness). 

8. Use a question and answer format to establish and meet an information need. 
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Group Effectiveness Skills 

All those at Training Level I plus: 

3. Express opinions and judgments using feelings, intuition, logic, and experience. 

4. Actively support group processes. 
Problem Solving and Analytical Thinking 
All those at Training Level I plus: 

2. Apply information to locate a problem or malfunction or to decide upon a course of 
action. 

Training Level III 

Communication Skills 

All those at Training Levels I and II plus: 

9. Recognize dominant communication styles by how they are manifested, and 
understand how they affect the message being communicated. 

10. Follow a line of thought as it develops among several speakers. 

11. Recognize shifts in communication purposes. 
Group Effectiveness Skills 

All those at Training Levels I and II plus: 

5. Support a group process by applying knowledge of group dynamics. 

6. Manage conflict on the job by using conflict resolution skills. 

7. Serve as a fully functioning member of a process improvement team using team task 
and maintenance skills. 

Problem Solving and Analytical Thinking 

All those at Training Levels I and II plus: 
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3. Recognize and utilize the concept of estimation in solving problems. 

4. Develop and evaluate ideas deductively (applying a general principle to a specific 
situation by using analogy or generalization to form a hypothesis, then exploring and 
testing its implications, and finally, recommending or not recommending its 
adoption). 

5. Construct and utilize graphic displays of information (e.g., histograms, linear graphs, 
circle charts). 

6. Utilize a variety of analytical techniques, such as Pareto (80-20 rule) and Fishbone 
(Ishekawa chart) methods of analysis. 

7. Disaggregate data from a conceptual model, flow chart, or schematic to constituent 
parts. 

8. Use statistical control charts to monitor normal variation in manufacturing processes 
and to identify abnormal fluctuation in those processes when key processing variables 
do not fall within limits of acceptable variation. 

#■ 

Computer Literacy 

All those at Training Levels I and II plus: 

1 . Read and comprehend screen displays and distinguish among key variables in those 
displays. 

2. Enter data using a computer keyboard or a punch pad. 

3. Using defined function keys and written procedures, move between and among screen 
displays to find appropriate data. 

Training Level IV 

Communication Skills 

All those at Training Levels I, II, and III plus: 

12. Determine underlying assumptions and biases. 

13. Evaluate messages in terms of credibility, usefulness, and appropriateness. 

14. Provide spoken directions in clear, unambiguous terms, displaying responsive and 
appropriate language behavior. 
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15. Determine that messages (directions) have been understood. 

16. Utilize all five fundamental skills of listening (listening for content, listening to 
converse, listening for long-term contexts, listening for emotional meaning, and 
listening to follow directions). 

Group Effectiveness Skills 

All those at Training Levels I, II, and III plus: 

8. Follow a line of thought as it develops among several speakers or participants of a 
group to keep the group focused. 

9. Place information or individual processes in the "big picture." 

10. Exercise effective leadership when in groups or teams. 

1 1 . Use a variety of tactical or operational planning techniques to structure and focus the 
activities of a team assigned to a project or seeking a goal or result. 

Problem Solving and Analytical Thinking 
All those at Training Levels I, II, and III plus: 

9. Identify opportunities for improvement in the manufacturing process, and develop and 
evaluate how these improvements can be implemented and what they would cost. 

10. Use a variety of analytical tools to develop and evaluate solutions to problems. 

11. Make inferences from empirical data (use inductive reasoning by moving from the 
specific facts to a general principle). 

12. Read and construct process flow charts. 

13. Integrate diverse or disaggregated data into a single conceptual model. 
Computer Literacy 

All those at Training Levels I, II, and III plus: 

4. Identify potential problems in the production process through the review of data on 
process monitor screen displays, and control key manufacturing processes by 
manipulation of data variables through keyboard or punchpad access to screen 
displays. 
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Operate a microcomputer both through a menu system as well as through its native 
operating system such as MS-DOS. 

Use word processing and spreadsheet software. 
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EVALUATION OF OFF-THE-SHELF LEARNING SYSTEMS 



The top nine integrated learning systems based on expert advice and CertainTeed project 
team analysis were: 

1 . Principles of the Alphabet Literacy System (PALS) — developed and marketed by 
IBM. 

2. PLATO — originally developed by Control Data Corporation, now marketed by the 
Roach Organization. 

3. CCC Learning System — developed and being marketed by the Computer Curriculum 
Corporation. 

4. Comprehensive Competencies Program (CCP) - originally developed by the 
Remediation and Training Institute (RTI), now being marketed by U.S. Basics. 

5. Sequential Training for Adult Reading (STAR) -» developed and being marketed by 
Hartley Courseware. 

6. Basic Academic Skills for Employment (BASE) - developed and being marketed by 
Educational Technologies. 

7. Job Skills Educational Program (JSEP) — originally developed by the U.S. Army, 
now being marketed by Loral Corporation. 

8. Compris Courseware — developed and being marketed by Compris Corporation of 
Canada. 

9. INVEST — developed and being marketed by Josten Learning. 
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PRECISION STRIKE TRAINING INSTRUCTION PROGRAM 



At CertainTeed, program participants received a minimum of 48 hours of instruction. This 
instruction was delivered in 16 three-hour sessions over a period of four months. Based on 
the results of the pre-tests, each individual was assigned computer lessons and workbook 
lessons. All program participants were required to attend the classroom sessions. All 
assignments were completed at the training center. The participants were strongly 
encouraged to work on their lessons when the Vance-Granville Community College instructor 
was on duty in the center and available to assist them. Most of the instructional hours were 
provided in this manner. However, a few instructional hours were undertaken at time when 
the instructor was not on duty. Program participants were not allowed to take the workbooks 
home. Several of the participants requested and received supplemental materials to work on 
at home. Hours spent on supplementary materials were not recorded. 

As part of the agreement to participate in this project, each employee agreed to complete the 
assigned computer and workbook lessons and attend all of the classroom sessions. The 
computer and workbook lessons were self paced. Each participant agreed to spend at least 
three hours each week working on their assigned lessons. Several of the participants spent 
more than the minimum. Individuals who mastered all of the assigned lessons were allowed 
to work on additional lessons that were directly related to the literacy skills required in their 
jobs. 

All program participants except one were able to master all lessons assigned during the 
duration of the project. 

In a lean manufacturing environment, the scheduling of training is always a problem. This is 
particularly true when the manufacturing process is continuous. Self-paced learning tools 
such as workbooks and computer assisted instruction allow for a degree of flexibility that is a 
must in a lean manufacturing environment. Workers may work on their assigned lessons at 
their convenience and at their own pace. 

One very important dimension for this project is the impact of shift changes. CertainTeed 
operates three shifts, 24 hours a day, five to six days a week. In addition, the work 
schedules are based on four nine-hour days or three 12-hour days. Employees were 
encouraged to participate in the instruction prior to and after shift. This seemed to pose no 
problem to workers who were not on the 12-hour shift. At the inception of the Precision 
Strike pilot project, plant management felt that those employees participating in the pilot 
project would not need to go to 12-hour shifts. However, shortly after the start of the 
project, pilot project participants did in fact move to 12-hour shifts. In addition, they also 
had to work third shifts. This move negatively impacted the morale of the project 
participants as well as their willingness to ^ome to the center prior to and after shift. Most 
of the participants opted to come on their days off. 
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In general, when looking at the Precision Strike program as a whole, individual participation 
levels dropped significantly when shifts were rotated. The moves from first to second to 
third shift upset individual worker routines and decreased the likelihood of continuous on- 
going participation. Many workers going from first to second and second to third shifts 
would drop out of program participation for the duration of the shift change. After shift 
rotation was over (usually about two weeks) the workers would come back and try to catch 
up. 
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EVALUATION DESIGN PROCEDURES 



For the Precision Strike evaluation, the general procedures used in applying the non- 
randomized control group pre-test/post-test design were: 

1) Experimental groups (Precision Strike participants) and control groups (norj- 
participants) were studied. These groups constituted a naturally (voluntarily) 
occurring collection of persons. Subjects were not assigned to groups randomly. 

2) Only the experimental group was exposed to the training. 

3) Groups were tested on the dependent variable Tl, for example, learning gains as 
measured by the Test of Adult Basic Education. 

4) The mean pre-test score was found for both groups. 

5) Groups were tested on the dependent variable T2, and mean post-test scores were 
found for both groups. 

6) Differences between Tl and T2 means for each group separately were found. 

7) Differences were compared to determine whether application of Precision Strike 
training was associated with a change favoring the experimental group over the 
control group (which was not exposed to training). 

8) Appropriate statistical tests were applied to determine whether differences in scores 
were significant — that is, if the difference was large enough to reject the null 
hypothesis that the difference was simply due to chance. 
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PRECISION STRIKE TRAINING FINAL REPORT 



INTRODUCTION 
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In 1992, Vance-Granville Community College, in partnership with CertainTeed 
Corporation and the National Alliance of Business, received a grant from the National 
Institute for Literacy to demonstrate Precision Strike training. Precision Strike training is 
designed to close the gaps between the current status of a company's workforce and where 
that workforce needs to be to compete successfully in global markets. Precision Strike was 
started as a pilot within a pilot program, Skills and Knowledge in Lifelong Learning 
(S.K.I.L.L.). S.K.I.L.L. involves customized, computerized lessons in basic skills, one-on-one 
tutoring, workbooks, and a few generally one-session classes. Precision Strike included 
S.K.I.L.L. activities but was enhanced by special classes and functional context workbooks 
customized for each participant's job. 

The primary product of this project is a guidebook for businesses that want to implement 
functional context workplace literacy programs (basic skills in the context of the 
workplace). The guidebook, Precision Strike Training in Lean Manufacturing: A Workplace 
Literacy Guidebook, not only gives step-by-step directions for initiating a Precision Strike 
workplace literacy program, but also relates CertainTeed's experiences implementing and 
evaluating the program. 



PURPOSE 



This report focuses on the formative and summative outcomes of CertainTeed's Precision 
Strike training program and activities not included in the mid-year report to the National 
Institute for Literacy. It is designed to serve as a companion to Precision Strike Training in 
Lean Manufacturing: A Workplace Literacy Guidebook, which provides information on most 
of the activities carried out as part of the Precision Strike project. 



ORGANIZATION 



This report has four major sections: 

• A summary of the activities carried out as part of the Precision Strike training 
program; 

• A summary of participation in the S.K.I.L.L. and Precision Strike programs; 

• A discussion of how S.K.I.L.L. and Precision Strike were evaluated and the results of 
those evaluations; and 

• A discussion of future activities emanating from the Precision Strike program. 



In all cases, this report limits itself to information not already contained in either the mid- 
year report to the National Institute for Literacy or the Precision Strike training guidebook. 



SUMMARY OF ACTIVITIES 



The following activities were carried out as part of the Precision Strike training program 
funded by the National Institute for Literacy: 

1. Conducted a literacy task analysis of ten key manufacturing jobs in an intact crew. 
A literacy task analysis was previously conducted in March, 1991, for all tasks for 17 
jobs. However, since several of these jobs had changed significantly since the last 
analysis, critical tasks for each were identified and analyzed again to update existing 
information, (Covered in the guide.) 

2. Updated the readability level of new, job-related reading material. (Covered in the 
guide.) 

3. Developed instructional materials for the functional context basic skills program 
based on the literacy task analysis. Functional context workbooks were developed 
for each of the ten crew members participating in the Precision Strike project. The 
preferred mode of delivery was to allow the workers time in the S.K.I.L.L. center to 
work on lessons while Vance-Granville Community Co lieu e instructors were 
available to assist them. (Covered in the guide.) 

4. Developed criterion-referenced assessment tools based ..he key elements of the 
instructional materials. These assessment tools were used \o measure the workers' 
skills before and after training and to determine the instruction each worker would 
require. (Covered in the guide). 

5. Provided 720 instructional hours over a four-month period during which employees 
worked with staff from Vance-Granville Community College in small groups using 
the functional context curriculum developed through this project. In addition to the 
workbook, instructors provided creative exercises and simulations of job scenarios 
that reinforced and supported the "lean manufacturing" concepts to which workers 
were being exposed on the plant floor. Instructors worked flexible schedules to 
provide each employee a minimum of three hours of training per week over the 
Precision Strike four-month training period. (Covered in the guide and this report.) 

6. Developed performance measures that are directly related to improved literacy skills. 
(Covered in the guide and in this report.) 

7. Analyzed the productivity returns to Precision Strike training by comparing 
performance measures for an experimental and comparison, group. (Covered in this 
report) 
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8. Produced a guidebook designed for manufacturing workplace literacy practitioners. 
(The guide serves this purpose.) 



Will disseminate the guidebooks to a manufacturing and community college 
audience and through the National Alliance of Business' network with trade 
associations and professional organizations. (Covered in this report.) 



PARTICIPATION IN S.K.LL.L. AND PRECISION STRIKE 
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S.K.LL.L. activities were incorporated in the functional context Precision Strike pilot 
program. Consequently, the participation rates in S.K.LL.L. are discussed here. The 
participation rates below reflect the 18-month S.K.LL.L. pilot program. The Training 
Director at the Oxford CertainTeed plant estimates that 70 to 75 percent of the plants then 
200 workers participated in S.K.LL.L. at least once or twice, and 35 to 40 percent came on a 
regular basis. 

The participation rates of the experimental C crew and the comparison A and B crews 
during the first 18 months of S.K.LL.L. are shown in the following box: 





Participation Rates of the Experimental 
and Comparison (A, B, and C) Crews 




A Crew 
Number 
Number 


of average S.K.LL.L. hours per crew member 
of total S.K.LL.L. hours for the whole crew 


19.4 
252.0 


Number 
member 


of average basic skill training hours per crew 


4.1 


B Crew 
Number 
Number 


of average S.K.LL.L. hours per crew member 
of total S.K.LL.L. hours for the whole crew 


106.0 
1484.0 


Number 
member 


of average basic skill training hours per crew 


52.0 


C Crew 
Number 
Number 


of average S.K.LL.L. hours per crew member 
of total S.K.LL.L. hours for the whole crew 


99.7 
1396.0 


Number 
member 


of average basic skill training hours per crew 


35.3 
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The chart shows that the C and B crews spent considerably more lime participating in 
S.K.LL.L than the A crew. Later we will show that these increases in training are matched 
with both learning gains and improvement in crew performance on "cost driver" variables 
such as run rate and operational delay. 

EVALUATION OF S.K.LL.L AND PRECISION STRIKE mmmmmfflfflmfflfflfflfflfflffimm 
The National Alliance of Business Evaluation 

The National Alliance of Business' evaluation of the Precision Strike project consisted of: 

Three site visits to CertainTeed's Oxford, North Carolina, Roofing Products Group 
Manufacturing Plant; 

One site visit to CertainTeed's Valley Forge, Pennsylvania, headquarters; 

A focus group of Vance-Granville Community College literacy instructors who had 
received training in functional context workplace literacy from the project 
consultant; 

A focus group of the C crew members, which is described at length later in this 
report; 

• Telephone interviews with project staff; 

• Analysis of the results of pre- and post-tests; and 

• Analysis of some key productivity variables. 

The results of two of the site visits and the focus group with Vance-Granville instruaors 
were described in the mid-year report to the National Institute for Literacy and are not 
discussed here This report includes information from the remainder of the evaluation 
components that are not included in the guidebook. 

Third Site Visit to Oxford Plant 

During this visit, NAB staff met with the project consultant and conducted a focus group 
with the C crew (described below). 5 v 

Visit to CertainTeed Valley Forge Headquarters 

During this visit, the NAB evaluator met with Bob Salstrand, Corporate Training Manager, 
as part of the effort (which included telephone calls to other key players) to find out why 
CertainTeed invested in S.K.I.L.L. and Precision Strike. According to Salstrand, Michel 
Besson, President of Saint Gobain (CertainTeed's parent company) saw himself as an 
Education President", and as such felt that he needed to invest in his employees' skills 



Salstrand himself was convinced that the programs were needed when workers could 
describe their jobs, but could not read their job descriptions. 

C Crew Members Focus Group 

In embarking on the S.K.I.L.L. and Precision Strike projects, CertainTeed involved workers 
in the Steering Committee from the start. The company was committed to improving 
workers* literacy skills off the job as well as on the job. In fact, family members were 
allowed to use the S.K.I.L.L. center. To examine what Precision Strike meant to its 
participants - the C crew - NAB evaluators conducted a focus group of C crew members. 
The findings are presented here. 

Purpose. A focus group of C crew members was held on May 6, 1993, to 
determine their opinions about the Precision Strike training which they had 
completed approximately two months earlier. 

The focus group addressed two major areas: how the Precision Strike training was 
helpful on and off the job, and how the program shoiYt! V 2' expanded. There was 
considerable consensus on most of the issues discussed. 

Methodology. The focus group was held in the conference room of CertainTeed's 
Oxford plant from 6:30 a.m. to 7:30 a.m. on May 6. Seven of the ten members of 
C crew who participated in Precision Strike attended the focus group. The focus 
group leader posed a question, and then requested a response from each participant 
in turn. A second evaluator recorded the responses. 

It should be noted that the new plant manager, Ed Foster, came in and sat at the 
back of the room midway through the session. His presence did not appear to 
affect the answers given by the participants. 

Focus Group Responses. The NAB facilitator posed four questions to the focus 
group participants. These were: 

• How did you find Precision Strike helpful on the job? 

• How do you use your new skills off the job? 

• How could Precision Strike be improved? 

• Should the program be expanded, and if so, what incentives are needed? 

In addition, focus group participants were asked to comment on motivation and 
direction of participating crew members and incentives to encourage participation. 
They also raised a few issues of their own. 

• How did you find Precision Strike helpful on the job? The participants 
were unanimous in saying that the program helped their math skills on the 
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job. Several specifically mentioned that their knowledge of fractions and/or 
decimals had improved. C crew members also mentioned learning about 
graphs and bar charts, as well as how to use a tape measure. 

Three crew members noted that they now paid more attention to detail and 
accuracy in the performance of job tasks. 

One crew member noted that Precision Strike brought back "dormant 
skills," and another that although he didn't need ll;e skills for his job, 
Precision Strike helped him to more fully understand the jobs of his co- 
workers. 

Only one respondent mentioned that his reading improved. Crew members 
do not read much on the job, and may be hesitant to admit to difficulty in 
reading job-related materials. 

Writing was not mentioned, although some participants would be expected 
to write work orders and to write in the logbook. However, during 
individual interviews conducted in January, some of the respondents did say 
their writing had improved. 

Crew members noted that the class on "Working" reduced stereotyping by 
getting crew members to concentrate on conditions or situations rather 
than individual people. 

How do you use your new skills off the job? A ! chough reading was not 
generally reported as a useful skill on the job, several participants reported 
that their leisure reading or ability to read to their children had improved. 
One noted that he read more newspaper articW i.ian before, and another 
that he now stopped to read magazine and newspaper articles rather than 
skimming them. A third said he could comprehend and remember what he 
read better. 

Others mentioned that since the training, they were more able to use the 
computer, help their children with homework, and assemble things (because 
their ability to measure things and read directions had improved). One said 
he felt more comfortable speaking after having participated in Precision 
Strike. 

In addition to reading to children and helping them with homework, one 
crew member noted that being in class himself had helped his relationship 
with his son's teacher. 

How could Precision Strike be improved? One member suggested that the 
workbooks should have been written with input from Line One operators 
(which includes the C as well as the A and B crews) instead of the Line 
Two operators. Although the training manager noted that Line Two 



included all skills needed for Line One as well as ethers, this indicates that 
at least one C crew member would have liked to Lave greater input into the 
development process. 

The crew thought that there should have been more formal classes, 
particularly those dealing with personal relationships such as the "Working" 
class. The "Working" class was popular and some crew members thought it 
should be offered to everyone in the plant. 

While one crew member would have liked to learn more about everyone's 
jobs, another thought the training should have been more individualized. 
He noted that hydraulics and pneumatics are relevant to some jobs but not 
to others. 

Two of the crew members would have liked more work on the computer. 
However, another remarked that he wanted more classes since it was the 
teacher who brought the computer to life. Two noted the importance of 
having the instructor right there to answer questions. 

Should the program be expanded? The ere v. numbers all agreed that the 
program should be expanded. Their reasons differed, but they all supported 
broadening Precision Strike. One noted it would nelp product quality, and 
another that it would make the company" more profitable. None of the 
participants suggested that the program should be mandatory. In fact, one 
cautioned that it should remain voluntary. 

Motivation and direction of participating crew members. Members of C 
crew could not answer whether they were more motivated to participate in 
Precision Strike than members of all other Oxford crews, because only A, 
B, and C crews on Line One were invited to take part in the program. 
However, they did think that their experiences in the program actually 
made them more motivated to continue and expand their participation. 
C crew members suggested that the best way to get and keep people 
motivated to participate in the program was to have the, program support 
people's self esteem. In other words, the program should not make 
participants feel unintelligent and should involve them as full contributing 
members in program development and operations. 

Finally, C crew members indicated that there was now an interest "on the 
floor" in Precision Strike, and that they could hAp Mike O'Brian, the 
training director, build on that interest and "sell" the program. 

What should the incentives be to encourage workers to participate? 
Several crew members emphasized that paying straight time rather than 
"half time" (one hour for every two worked) would work better than any 
other kind of incentive. 
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Eight-hour shifts were mentioned as a way to attract more people to the 
program, since staying to work on Precision Strike after a 12-hour shift or 
coming in on one's day off is more of a burden than coming early or 
staying late after an eight-hour shift. 

The C crew members thought that some of the incentives used currently, 
such as certificates to restaurants, were also effective. 

• Other issues. One member thought that Precision Strike participants 
should only be made to take instruction that is job related (that is, related to 
an individual crew member's job) - except for lowCivel, non-functional 
context basic skills. If the curricula is not job related then it should be 
optional. 

The C crew members were in agreement that it was important for 
confidentiality to be maintained. For example, workers do not know what 
training levels others are working on because the workbooks have identical 
covers. 

Some C crew members thought that requirements for the program should 
be made clearer. There was some concern expressed that the "Working" 
class was first voluntary and then mandatory. In fact, it was never 
mandatory for Precision Strike, but it had always been mandatory for 
completing a training level. 

In Summary, the focus group members thought Precision Strike: 

• Had helped their math on the job and their reading off the job; 

• Should have included more classroom work, particularly classes emphasizing 
relationships, such as "Working", which they ail ?'aced highly; 

• Should have included more computer work; 

• Should be expanded to include more Oxford workers; 

• Should include incentives for participation, including being paid at straight 
time rather than at "half time"; 

• Should be offered to workers when they are on eight-hour shifts rather than 
on 12-hour shifts. 
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Telephone Interviews 

NAB staff conducted interviews with David Morman, the loaned executive to CertainTeed 
from the U.S. Department of Labor, who had recommended a workplace literacy program 
to the company, and with Bill Ricketts, then Vice President for Manufacturing for 
CertainTeed. Both individuals reinforced Salstrand's beliefs that the workplace literacy 
programs were valuable and Bill Ricketts added that the program?; provided a means for 
promoting from within. 

NAB staff also conducted a face-to-face interview and several telephone interviews with 
Mike O'Brian, Oxford's Training Director. During these sessions, he provided information 
on the evolution of Precision Strike, described at the end of this report. 

The Project Director Interview. Subsequent to the third visit to Ox ord, an 
interview was held with the Vance-Granville-based projeo director. From her 
viewpoint, Precision Strike is valuable and should be expaided. She noted that 
when there are problems on the line, the crew members ^ ho participated in 
training now start looking at their charts and logbooks and really attempt to solve 
problems. 

If Precision Strike is expanded in Oxford, the project clir^ tor believes it should be 
mandatory for all workers. She thinks the biggest obstacle to setting up a 
mandatory program would be setting up schedules and de.ding with workers' 
objections to participating in training after shift. 

According to the project director, the Precision Strike project has generated interest 
in functional context workplace literacy in other industries. Since late December, 
1992, Vance-Granville Community College has helped initiate functional context 
programs at: Hon Company, which makes office furniture and is training all new 
hires in shop math; Cochran Furniture, which is converting a 21-hour traditional 
classroom program to a functional context program; and Alpine Structures, which 
was in the process of conducting a literacy task analysis at the time of the project 
director interview. 

Analysis of Outcomes 

NAB's project consultant conducted an in-depth evaluation of the S.K.I.L.L. and Precision 
Strike programs for CertainTeed to determine whether the progmns should be continued. 
He compiled a data base with numerous variables. In addition, NAB staff conducted an 
evaluation on a smaller scale for the National Institute for Literacy. This evaluation 
examined some of CertainTeed's outcomes data, but focused on process issues. This was 
for two reasons: many of the outcomes are proprietary, and the CertainTeed data base was 
too large to easily manipviate. 

Performance Measures. CertainTeed used as performance measures the Test of 
Adult Basic Education (TABE), the Cloze test, and pre- and post-functional context 
tests customized for each crew position in the experimental and comparison crews. 
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CertainTeed conducted performance and productivity measurements that compared 
individuals or crews before participating in the S.K.I.L.L. program, during the 
program, and after the program. Most of CertainTeed's specific performance and 
productivity results are proprietary, but in general, the individuals and/or crews 
who had participated in the S.K.I.L.L. or Precision Strike projects for more hours 
tended to perform better than their counterparts with few-^r hours. (An example of 
run rate and coated waste follows.) The variables that CertainTeed looked at 
included: 

• Process monitoring system audits, 

• Pre- and post-test results for Precision Strike participants, 

• Post-test results for A, B, and C crew members, 

• Post Cloze test results for A, B, and C crew members, 

• Gains in TABE scores by crew and S.K.I.L.L. participation, 

• Responses to employee and supervisor surveys, 

• Finished product audit scores for selected quality tactors, 

• Run rates, 

• Coated waste, 

• Total waste, 

• Delay, 

• Operational delay, 

• Shipping errors, 

• Attendance rates, and 

• Overall performance ratings. 

Although details on many measures are proprietary, the following summarizes some 
of the results: 

• Both the B and C crews (with high levels of participation in S.K.I.L.L.) had 
higher attendance rates than the A crew. 

• Individuals with high S.K.I.L.L. participation rates had higher attendance 
rates than those with low levels of participation. 

• Before S.K.I.L.L., the C crew had the highest customer complaint rate of 
the three crews for Product A 1 . After S.K.I.L.L., they had the lowest 
customer complaint rate. 

Learning gains on pre- and post-tests and changes in two of the "cost driver" 
variables are discussed in the following section. 

Results - Returns to Precision Strike Training. This section compares 
performance measures between experimental and comparison groups. 

Analysis of the Precision Strike data revealed clear-cut learning gains on both the 



betters are used to represent CertainTeed products for proprietary reasons, 
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functional context pre- and post-tests, and on the Clo^ 1 t. The Precision Strike 
crew also performed better than the non-participating crews on some "cost driver" 
measures such as run rate and coated waste. For example. 

• The nine Precision Strike participants with scores on both the pre- and post- 
functional context tests had an average increase in their scores of 10.3 
percentage points; 

• The same participants had an average increase of 15.4 percentage points on 
the pre- and post-Cloze tests; 

• The scores of the Precision Strike participants on the post-functional 
context test averaged 81.7 percent, while the two non-participating crews 
had averages of 61.1 and 70.2 percent. This finding is significant because all 
three crews were similar on such characteristics as TABE scores, years of 
schooling, years of experience at the plant, and age. Therefore, it is likely 
that the differences in scores are due to Precision Strike training. 

• Precision Strike participants averaged 74 percent on the post-Cloze test, 
while the other crews averaged 60.3 and 41 percent. 



Percentage Changes in Selected Productivity Measures 
During Four-Month Periods Pre- and Post- 
Precision Strike Training 




Crew A 


Crew B 


Crew C 


Run Rate A 


+6.5% 


+ 9.3% 


+6.0% 


Run Rate B 


+ .81% 


+ .22% 


+4.2% 


Coated Waste A 


-20.0% 


-18.7% 


-29.7% 


Coated Waste B 


+ 3.1% 


-9.2% 


-8.1% 



The table above is a sample of the kind of "cost driver" data analysis that was 
presented to the CertainTeed Board of Directors by the National Alliance of 
Business project consultant on May 4, 1993. It shows changes in run rate and 
coated waste by crew for two roofing products, A and B, which represent grades of 
shingles. These data have been updated since the May 4 presentation so that the 
pre- (July to October 1992) and post- (March to June 1 .5?3) comparison periods are 
both four months long. 
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In examining this table, recall that A crew members received minimal S.K.LL.L. 
training, the B crew received extensive S.K.LL.L. training, and the C crew received 
S.K.LL.L. training as well as the special functional context Precision Strike training. 

The results are mixed. On the A product, the C crew had the smallest percentage 
increase in run rate (defined as squares produced as a percent of actual machine 
hours), but a spectacular decrease in coated waste - 29.7 percent - ten or more 
percentage points higher than the decreases experienced by the A and B crews. 

On the B product, the results are somewhat reversed. C brew's percentage increase 
in run rate is greater than that of the A and B ciews by a factor of five or more. 
Decreases in coated waste are just one percentage point lower than B crew's, and 
several points higher than for A crew. 

As noted above, these are but a small sample of the changes in the cost driver 
variables. However, given the positive changes for the C and B crews compared to 
the A crew, they indicate that there are real returns to both S.K.LL.L. and 
Precision Strike training. 



IN Jb A. 1 o I JcLl b 

Dissemination of the Guidebook 

The National Alliance of Business will disseminate the guidebook and project results 
through its normal dissemination routes, including its Annual Conference and on-going 
workshops and seminars, and through its extensive relationships with the business 
community and its contacts with national trade organizations and professional 
organizations such as the National Association of Manufacturers and the American 
Association of Community Colleges. 

The Future of Functional Context Training at Oxford and 3 her CertainTeed Roofing 
Products Group Plants 

A discussion with CertainTeed's Oxford plant Training Director in October, 1993, 
indicated that S.K.LL.L. training continues, but that a new form of training has evolved 
from the Precision Strike training. This training, known as job linked training, focuses on 
the job instead of on the crew. Training for the first job, the top slate operator, started in 
mid-October 1993. 

The Training Director selected the top slate operator position first because the job is 
critical and because the TABE scores among top slate operators v.-ere above average. He 
thought that higher TABE scores would make the training go more smoothly. This was 
desirable for the start of job linked training. 

While the Precision Strike training had functional context workbooks for each job, most of 
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the training targeted the whole crew. With job linked training, on the other hand, all of 
the training is directly linked to a particular job. Another difference from Precision Strike 
is that the training is mandatory. It will primarily be "on the clock", although for any one 
class, two of the six top slate operators will have to come in wheii they are normally off. 
However, they will be fully paid for this time. 

The top slate operator job linked training will include job linked classes on quality, 
communications skills, team training, problem solving, and Statistical Process Control 
(SPC) run charts. The basic skills training each individual has will be determined by his or 
her functional context pre-test. While the basic skills training may still include some 
generic training at the S.K.I.L.L. center, it will be followed by individualized work on a 
functional context basic skills workbook. 

The Training Director explained that job linked training is part of an approach to integrate 
functional context basic skills training with CertainTeed's technical training (which consists 
of job specific on-the-job training with designated trainers or mentors). In addition, job 
linked training will become part of the bidding system for new jobs. 

Once the top slate training is underway, the curriculum for the next job, finished product 
inspector, will be developed. Later, training will be developed for all the jobs in 
CertainTeed's Roofing Products Group. 

Job linked training also will be expanded beyond the Oxford plane. The Avery, Ohio, and 
Shakopee, Minnesota, Roofing Products Group plants have recent. y been approved for job 
linked training programs. Although the model has been modineo, the functional context 
core of Precision Strike training lives on at CertainTeed! 
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